FileStorageModel

java.lang.Object
- org.das2.fsm.FileStorageModel

```
public class FileStorageModel
extends java.lang.Object
```
Represents a method for storing data sets in a set of files by time. The client provides a regex for the files and how each group of the regex is interpreted as a time digit. The model can then be used to provide the set of files that cover a time range, etc. This new implementation uses a TimeParser object to more quickly process file names.

Field Summary

Fields
Modifier and Type	Field and Description
`protected Datum`	`implicitTimeDelta` limit on the length of files with startTimeOnly set.
`protected boolean`	`startTimeOnly` The filename time only contains the start time of the interval, the end of the interval is only bounded by the next file.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`cacheCleanup()` remove files that have been identified as old versions.
`boolean`	`containsFile(java.io.File file)` return true if the file came (or could come) from this FileStorageModel.
`static FileStorageModel`	`create(FileSystem root, java.lang.String template)` creates a FileStorageModel for the given template, which uses:
`static FileStorageModel`	`create(FileSystem root, java.lang.String template, java.lang.String fieldName, TimeParser.FieldHandler fieldHandler)` creates a FileStorageModel for the given template, but with a custom FieldHandler and field.
`java.lang.String[]`	`generateNamesFor(DatumRange range)` generate the names of the files that would cover this range.
`java.io.File[]`	`getBestFilesFor(DatumRange targetRange)` return the best files found for the range, without progress feedback.
`java.io.File[]`	`getBestFilesFor(DatumRange targetRange, ProgressMonitor monitor)` Get the files for the range, using versioning info ($v,etc).
`java.lang.String[]`	`getBestNamesFor(DatumRange targetRange, ProgressMonitor monitor)` return the names in the range, minding version numbers, or all available names if the range is null.
`static CacheTag`	`getCacheTagFor(FileStorageModel fsm, DatumRange range, java.io.File[] files)`
`static CacheTag`	`getCacheTagFor(FileStorageModel fsm, DatumRange range, java.lang.String[] names)`
`java.lang.String`	`getField(java.lang.String field, java.lang.String name)` return the field value for the given name.
`java.io.File`	`getFileFor(java.lang.String name)` download the file for the given name within the filesystem.
`java.io.File`	`getFileFor(java.lang.String name, ProgressMonitor monitor)` download the file for the given name within the filesystem.
`java.lang.String`	`getFilenameFor(Datum start, Datum end)` return a filename that would intersect the range.
`java.io.File[]`	`getFilesFor(DatumRange targetRange)`
`java.io.File[]`	`getFilesFor(DatumRange targetRange, ProgressMonitor monitor)` Download the files within the range.
`java.io.File[]`	`getFilesFor(java.lang.String[] names, ProgressMonitor monitor)` download the files for each of the given names within the filesystem.
`FileSystem`	`getFileSystem()` return the filesystem used to implement this.
`java.lang.String`	`getNameFor(java.io.File file)` Provides a way to recover the model name of a file.
`java.lang.String[]`	`getNamesFor(DatumRange targetRange)` return the names in the range, or all names if the range is null.
`java.lang.String[]`	`getNamesFor(DatumRange targetRange, ProgressMonitor monitor)` return the names in the range, or all available names if the range is null.
`FileStorageModel`	`getParent()` returns the parent or null if none exists.
`DatumRange`	`getRangeFor(java.lang.String name)` return the time range represented by this name.
`java.lang.String`	`getRepresentativeFile(ProgressMonitor monitor)` return a random file from the FSM, which can be used to represent a typical file.
`java.lang.String`	`getRepresentativeFile(ProgressMonitor monitor, java.lang.String childRegex)` return a random file from the FSM, which can be used to represent a typical file.
`java.lang.String`	`getRepresentativeFile(ProgressMonitor monitor, java.lang.String childRegex, DatumRange range)` Return a random file from the FSM, which can be used to represent a typical file.
`protected static java.lang.String`	`makeCanonical(java.lang.String template)` Autoplot introduced the dollar sign instead of the percent, because $ is more URI-friendly.
`DatumRange`	`quantize(DatumRange timeRange)` return the timerange that contains the given timerange and exactly contains a set of granules.
`void`	`setContext(DatumRange trdr)` set the datum range giving context to the files.
`static int`	`splitIndex(java.lang.String surl)` This returns the index splitting the static part of the filename from the templated part.
`java.lang.String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - startTimeOnly
```
protected boolean startTimeOnly
```
    The filename time only contains the start time of the interval, the end of the interval is only bounded by the next file.
  - implicitTimeDelta
```
protected Datum implicitTimeDelta
```
    limit on the length of files with startTimeOnly set. e.g. $Y$m$d_$(H,startTimeOnly)$M means that the files should be much less that one hour long
- Method Detail
  - getFileSystem
```
public FileSystem getFileSystem()
```
    return the filesystem used to implement this.
    
    Returns:
    
    filesystem
  - getRepresentativeFile
```
public java.lang.String getRepresentativeFile(ProgressMonitor monitor)
                                       throws java.io.IOException
```
    return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata to see what is available.
    
    Parameters:
    
    monitor - progress monitor in case a file must be downloaded.
    
    Returns:
    
    a reference to the file within the FileSystem, or null if one is not found.
    
    Throws:
    
    java.io.IOException
  - getRepresentativeFile
```
public java.lang.String getRepresentativeFile(ProgressMonitor monitor,
                                              java.lang.String childRegex)
                                       throws java.io.IOException
```
    return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata to see what is available.
    
    Parameters:
    
    monitor - progress monitor in case a file must be downloaded.
    
    childRegex - the parent must contain a file/folder matching childRegex
    
    Returns:
    
    a reference to the file within the FileSystem, or null if one is not found.
    
    Throws:
    
    java.io.IOException
  - getRepresentativeFile
```
public java.lang.String getRepresentativeFile(ProgressMonitor monitor,
                                              java.lang.String childRegex,
                                              DatumRange range)
                                       throws java.io.IOException
```
    Return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata to see what is available. This is introduced to support discovery, where we just need one file to get started. Before, there was code that would list all files, then use just the first one. This may return a skeleton file, but getFileFor() must return a result. This implementation does the same as getNames(), but stops after finding a file.
    
    Parameters:
    
    monitor - progress monitor in case a file must be downloaded.
    
    childRegex - the parent must contain a file/folder matching childRegex
    
    range - hint at the range where we are looking.
    
    Returns:
    
    null if no file is found
    
    Throws:
    
    java.io.IOException - if the file cannot be downloaded.
  - setContext
```
public void setContext(DatumRange trdr)
```
    set the datum range giving context to the files. For example, filenames are just $H$M$S.dat, and the context is "Jan 17th, 2015"
    
    Parameters:
    
    trdr - the context
  - getFilenameFor
```
public java.lang.String getFilenameFor(Datum start,
                                       Datum end)
```
    return a filename that would intersect the range. Note this file may not actually exist. This may be used to quantize the range. The template may not contain versions.
    
    Parameters:
    
    start -
    
    end -
    
    Returns:
  - generateNamesFor
```
public java.lang.String[] generateNamesFor(DatumRange range)
```
    generate the names of the files that would cover this range. This was taken from Autoplot's org.virbo.jythonsupport.Util. TODO: versioning, etc.
    
    Parameters:
    
    range - the time range to cover.
    
    Returns:
    
    the string names, each in the context of the filesystem.
  - quantize
```
public DatumRange quantize(DatumRange timeRange)
```
    return the timerange that contains the given timerange and exactly contains a set of granules. This needs to be synchronized because the timeParser.
    
    Parameters:
    
    timeRange - arbitrary time range
    
    Returns:
    
    list of file timeranges covering file input timeRange.
  - getNamesFor
```
public java.lang.String[] getNamesFor(DatumRange targetRange)
                               throws java.io.IOException
```
    return the names in the range, or all names if the range is null.
    
    Parameters:
    
    targetRange - range limit, or null.
    
    Returns:
    
    array of names within the system.
    
    Throws:
    
    java.io.IOException
  - getNamesFor
```
public java.lang.String[] getNamesFor(DatumRange targetRange,
                                      ProgressMonitor monitor)
                               throws java.io.IOException
```
    return the names in the range, or all available names if the range is null. This will list directories.
    
    Parameters:
    
    targetRange - range limit, or null.
    
    monitor -
    
    Returns:
    
    array of names within the system.
    
    Throws:
    
    java.io.IOException
  - getBestNamesFor
```
public java.lang.String[] getBestNamesFor(DatumRange targetRange,
                                          ProgressMonitor monitor)
                                   throws java.io.IOException
```
    return the names in the range, minding version numbers, or all available names if the range is null. This will list directories.
    
    Parameters:
    
    targetRange - range limit, or null.
    
    monitor -
    
    Returns:
    
    array of names within the system.
    
    Throws:
    
    java.io.IOException
  - getCacheTagFor
```
public static CacheTag getCacheTagFor(FileStorageModel fsm,
                                      DatumRange range,
                                      java.lang.String[] names)
```
  - getCacheTagFor
```
public static CacheTag getCacheTagFor(FileStorageModel fsm,
                                      DatumRange range,
                                      java.io.File[] files)
```
  - getFilesFor
```
public java.io.File[] getFilesFor(DatumRange targetRange)
                           throws java.io.IOException
```
    Throws:
    
    java.io.IOException
  - getBestFilesFor
```
public java.io.File[] getBestFilesFor(DatumRange targetRange)
                               throws java.io.IOException
```
    return the best files found for the range, without progress feedback.
    
    Parameters:
    
    targetRange -
    
    Returns:
    
    Throws:
    
    java.io.IOException
  - cacheCleanup
```
public void cacheCleanup()
```
    remove files that have been identified as old versions.
  - getRangeFor
```
public DatumRange getRangeFor(java.lang.String name)
```
    return the time range represented by this name.
    
    Parameters:
    
    name - like 2013-10-31
    
    Returns:
    
    the timerange representing the day 2013-10-31
    
    Throws:
    
    java.lang.IllegalArgumentException - if the name is not part of the FileStorageModel.
  - getField
```
public java.lang.String getField(java.lang.String field,
                                 java.lang.String name)
```
    return the field value for the given name. For example, if the spec is $Y/$m/$d/$Y$m$d_v$(v,sep).dat and the name matched is 2014/04/04/20140404_v2.3.dat then calling this for the field "v" would result in "2.3" This should not be used to retrieve fields that are components of the time range, such as $Y or $m.
    
    Parameters:
    
    field - field, for example "v"
    
    name - name, for example 2014/04/04/20140404_v2.3.dat
    
    Returns:
    
    the field value, for example, "2.3" when the spec is $Y/$m/$d/$Y$m$d_v$v.dat
  - containsFile
```
public boolean containsFile(java.io.File file)
```
    return true if the file came (or could come) from this FileStorageModel.
    
    Parameters:
    
    file - the file
    
    Returns:
    
    true if the file came (or could come) from this FileStorageModel.
  - getNameFor
```
public java.lang.String getNameFor(java.io.File file)
```
    Provides a way to recover the model name of a file. The returned File from getFilesFor can be anywhere, so it would be good to provide a way to get it back into a FSM name. For example, a filesystem might download the remote file to a cache directory, which is the File that is provided to the client, sometimes the client will need to recover the name of the corresponding FileObject, so this maps the File back to the name.
    
    Parameters:
    
    file - the file
    
    Returns:
    
    the canonical name of the file.
  - getFileFor
```
public java.io.File getFileFor(java.lang.String name)
                        throws java.io.IOException
```
    download the file for the given name within the filesystem.
    
    Parameters:
    
    name - the name within the filesystem.
    
    Returns:
    
    null or a local file which can be opened.
    
    Throws:
    
    java.io.IOException
  - getFileFor
```
public java.io.File getFileFor(java.lang.String name,
                               ProgressMonitor monitor)
                        throws java.io.IOException
```
    download the file for the given name within the filesystem.
    
    Parameters:
    
    name - the name within the filesystem.
    
    monitor - monitor for the download.
    
    Returns:
    
    null or a local file which can be opened.
    
    Throws:
    
    java.io.IOException
  - getFilesFor
```
public java.io.File[] getFilesFor(java.lang.String[] names,
                                  ProgressMonitor monitor)
                           throws java.io.IOException
```
    download the files for each of the given names within the filesystem.
    
    Parameters:
    
    names - array of names within the filesystem
    
    monitor - monitor for the downloads.
    
    Returns:
    
    local files that can be opened.
    
    Throws:
    
    java.io.IOException - during the transfer
  - getFilesFor
```
public java.io.File[] getFilesFor(DatumRange targetRange,
                                  ProgressMonitor monitor)
                           throws java.io.IOException
```
    Download the files within the range. This might catch a bad link where getNamesFor does not.
    
    Parameters:
    
    targetRange - range limit, or null if no constraint used here.
    
    monitor - the monitor
    
    Returns:
    
    a list of files that can be opened.
    
    Throws:
    
    java.io.IOException
  - getBestFilesFor
```
public java.io.File[] getBestFilesFor(DatumRange targetRange,
                                      ProgressMonitor monitor)
                               throws java.io.IOException
```
    Get the files for the range, using versioning info ($v,etc).
    
    Parameters:
    
    targetRange - range limit, or null if no constraint used here.
    
    monitor - the monitor
    
    Returns:
    
    a list of files that can be opened.
    
    Throws:
    
    java.io.IOException
  - getParent
```
public FileStorageModel getParent()
```
    returns the parent or null if none exists. None will exist when there are no more wildcards. (/home/foo/$Y/$m-$d.dat has parent /home/foo/$Y which has parent null.)
    
    Returns:
    
    parent or null.
  - makeCanonical
```
protected static java.lang.String makeCanonical(java.lang.String template)
```
    Autoplot introduced the dollar sign instead of the percent, because $ is more URI-friendly. Switch to this if it looks appropriate. NOTE: this does not correct commas in the qualifiers section. TODO: fix this!
    
    Parameters:
    
    template, - e.g "/%Y/%m%d.dat"
    
    Returns:
    
    "$Y/$m$d.dat"
  - splitIndex
```
public static int splitIndex(java.lang.String surl)
```
    This returns the index splitting the static part of the filename from the templated part. For example, http://autoplot.org/data/C1_CP_EDI_EGD__$Y$m$d_V$v.cef is split into:
```
http://autoplot.org/data/
C1_CP_EDI_EGD__$Y$m$d_V$v.cef
```
    and http://emfisis.physics.uiowa.edu/Flight/RBSP-A/Quick-Look/$Y/$m/$d/rbsp-a_magnetometer_4sec-gsm_emfisis-Quick-Look_$Y$m$d_v$(v,sep).cdf:
```
http://emfisis.physics.uiowa.edu/Flight/RBSP-A/Quick-Look/
$Y/$m/$d/rbsp-a_magnetometer_4sec-gsm_emfisis-Quick-Look_$Y$m$d_v$(v,sep).cdf
```
    This new version uses regexs and is more complete than versions found in Autoplot, and they should eventually use this instead. Note the Autoplot one returns the index of the last /, whereas this returns that index plus one.
    
    Taken from Autoplot's AggregatingDataSourceFactory, where Autoplot just has a URI and needs to get a file list. See also org/autoplot/pngwalk/WalkUtil.java splitIndex, which also allows wildcards like *.
    Parameters:
    
    surl - a string like http://autoplot.org/data/C1_CP_EDI_EGD__$Y$m$d_V$v.cef
    
    Returns:
    
    an integer indicating the split index, so that surl.substring(0,i) includes the slash.
  - create
```
public static FileStorageModel create(FileSystem root,
                                      java.lang.String template)
```
    creates a FileStorageModel for the given template, which uses:
```
%Y-%m-%dT%H:%M:%S.%{milli}Z";
    %Y  4-digit year
    %m  2-digit month
    %d  2-digit day of month
    %j  3-digit day of year
    %H  2-digit Hour
    %M  2-digit Minute
    %S  2-digit second
    %v  best version by number  Also %(v,sep) for 4.3.2  or %(v,alpha)
    %{milli}  3-digit milliseconds 
```
    product_$(o,id=ftp://stevens.lanl.gov/pub/projects/rbsp/autoplot/orbits/rbspa_pp).png
    Parameters:
    
    root - FileSystem source of the files.
    
    template - describes how filenames are constructed. This is converted to a regular expression and may contain regex elements without groups. The string may contain $ instead of percents as long as there are no percents in the string.
    
    Returns:
    
    a newly-created FileStorageModel.
  - create
```
public static FileStorageModel create(FileSystem root,
                                      java.lang.String template,
                                      java.lang.String fieldName,
                                      TimeParser.FieldHandler fieldHandler)
```
    creates a FileStorageModel for the given template, but with a custom FieldHandler and field.
    
    Parameters:
    
    root - FileSystem source of the files.
    
    template - describes how filenames are constructed. This is converted to a regular expression and may contain regex elements without groups. The string may contain $ instead of percents as long as there are no percents in the string.
    
    fieldName - custom field name
    
    fieldHandler - TimeParser.FieldHandler to call with the field contents.
    
    Returns:
    
    a newly-created FileStorageModel.
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object

Class FileStorageModel

Field Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

startTimeOnly

implicitTimeDelta

Method Detail

getFileSystem

getRepresentativeFile

getRepresentativeFile

getRepresentativeFile

setContext

getFilenameFor

generateNamesFor

quantize

getNamesFor

getNamesFor

getBestNamesFor

getCacheTagFor

getCacheTagFor

getFilesFor

getBestFilesFor

cacheCleanup

getRangeFor

getField

containsFile

getNameFor

getFileFor

getFileFor

getFilesFor

getFilesFor

getBestFilesFor

getParent

makeCanonical

splitIndex

create

create

toString