[gpfsug-discuss] AFM Prefetch Missing Files

[Wilson, Neil]

We recently migrated several hundred TB from an Isilon cluster to our GPFS cluster using AFM using NFS gateways mostly using 4.2.2.2 , the main thing we noticed was that it would not migrate empty directories - we worked around the issue by getting a list of the missing directories and running it through a simple script that cd's into each directory then lists the empty directory. I didn't come across any issues with symlinks not being prefetched, just the directories. 

Regards
Neil Wilson

-----Original Message-----
From: gpfsug-discuss-bounces at spectrumscale.org [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Peter Childs
Sent: 18 May 2017 10:08
To: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] AFM Prefetch Missing Files

Further investigation and checking says 4.2.1 afmctl prefetch is missing empty directories (not files as said previously) and noted by the update in 4.2.2.3. However I've found it is also missing symlinks both dangling (pointing to files that don't exist) and not.

I can't see any actual data loss which is good.

I'm looking to work around this with

find /data2/$fileset -noleaf \( \( -type d -empty \) -o \( -type l \) \) -printf "%p -> %l\n"

My initial testing says this should work. (/data2/$fileset is the destination "cache" fileset)

It looks like this should catch everything, But I'm wondering if anyone else has noticed any other things afmctl prefetch misses.

Thanks in advance

Peter Childs

On 16/05/17 10:40, Peter Childs wrote:
> I know it was said at the User group meeting last week that older 
> versions of afm prefetch miss empty files and that this is now fixed 
> in 4.2.2.3.
>
> We are in the middle of trying to migrate our files to a new 
> filesystem, and since that was said I'm double checking for any 
> mistakes etc.
>
> Anyway it looks like AFM prefetch also misses symlinks pointing to 
> files that that don't exist. ie "dangling symlinks" or ones that point 
> to files that either have not been created yet or have subsequently 
> been deleted. or when files have been decompressed and a symlink 
> extracted that points somewhere that is never going to exist.
>
> I'm still checking this, and as yet it does not look like its a data 
> loss issue, but it could still cause things to not quiet work once the 
> file migration is complete.
>
> Does anyone else know of any other types of files that might be missed 
> and I need to be aware of?
>
> We are using 4.2.1-3 and prefetch was done using "mmafmctl prefetch" 
> using a gpfs policy to collect the list, we are using GPFS 
> Multi-cluster to connect the two filesystems not NFS....
>
> Thanks in advanced
>
>
> Peter Childs
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss