org.das2.qds.util.AsciiParser

Class for reading ASCII tables into a QDataSet. This parses a file by breaking it up into records, and passing the record off to a delegate record parser. The record parser then breaks up the record into fields, and each field is parsed by a delegate field parser. Each column of the table has a Unit, field name, and field label associated with it. Examples of record parsers include DelimParser, which splits the record by a delimiter such as a tab or comma, RegexParser, which processes each record with a regular expression to get the fields, and FixedColumnsParser, which splits the record by character positions. Example of field parsers include DOUBLE_PARSER which parses the value as a double, and UNITS_PARSER, which uses the Unit attached to the column to interpret the value. When the first record with the correct number of fields is found but is not parseable, we look for field labels and units. The skipLines property tells the parser to skip a given number of header lines before attempting to parse the record. Also, commentPrefix identifies lines to be ignored. In either the header or in comments, we look for propertyPattern, and if a property is matched, then the builder property is set. Two Patterns are provided NAME_COLON_VALUE_PATTERN and NAME_EQUAL_VALUE_PATTERN for convenience. Adapted to QDataSet model, Jeremy, May 2007.

NAME_COLON_VALUE_PATTERN

pattern for name:value.

NAME_EQUAL_VALUE_PATTERN

pattern for name=value.

PROPERTY_FIELD_NAMES

PROPERTY_FILE_HEADER

PROPERTY_FIRST_RECORD

PROPERTY_FIELD_PARSER

DELIM_COMMA

DELIM_TAB

DELIM_WHITESPACE

UNIT_UTC

Convenient unit for parsing UTC times.

PROP_HEADERDELIMITER

DOUBLE_PARSER

parses the field using Double.parseDouble, Java's double parser.

UNITS_PARSER

delegates to the unit object set for this field to parse the data.

ENUMERATION_PARSER

uses the EnumerationUnits for the field to create a Datum.

PROP_VALIDMIN

PROP_VALIDMAX

getDelimParser

provide more control to external codes by providing a way to assert that an N-column delim parser should be used.

Parameters

Returns:

getFieldCount

return the number of fields in each record. Note the RecordParsers also have a fieldCount, which should be equal to this. This allows them to be independent of the parser.

Returns:

getFieldIndex

returns the index of the field. Supports the name, or field0, or 0, etc. returns -1 when the column is not identified.

Parameters

Returns:

getFieldLabels

return the labels found for each field. If a label wasn't found, then the name is returned.

Returns:

getFieldNames

return the name of each field. field0, field1, ... are the default names when names are not discovered in the table. Changing the array will not affect internal representation.

Returns:

getFieldUnits

return the units that were associated with the field. This might also be the channel label for spectrograms. In "field0(str)" or "field0[str]" this is str. elements may be null if not found.

Returns:

getFillValue

return the fillValue. numbers that parse to this value are considered to be fill. Note validMin and validMax may be used as well.

Returns:

getHeaderDelimiter

get the header delimiter

Returns:

getRecordParser

Getter for property recordParser.

Returns:

getRegexForFormat

Convert FORTRAN (F77) style format to C-style format specifiers.

Parameters

Returns:

See Also:

getRegexParser

return a regex parser for the given regular expression. Groups are used for the fields, for example getRegexParser( 'X (\d+) (\d+)' ) would parse lines like "X 00005 00006".

Parameters

Returns:

getRegexParserForFormat

see private TimeParser(String formatString, Map fieldHandlers), which is very similar.

• "%5d%5d%9f%s"
• "d5,d5,f9,a"

Parameters

Returns:

See Also:

getRichFields

returns the high rank rich fields in a map from NAME to LABEL. NAME:>fieldX< or NAME:>fieldX-fieldY<

Returns:

getValidMax

get the maximum value for any field.

Returns:

getValidMin

get the minimum valid value for any field.

Returns:

guessDelimParser

Returns:

guessFieldCount

return the field count that would result in the largest number of records parsed. The entire file is scanned, and for each line the number of decimal fields is counted. At the end of the scan, the fieldCount with the highest record count is returned.

Parameters

Returns:

guessSkipAndDelimParser

read in records, allowing for a header of non-records before guessing the delim parser. This will return a reference to the DelimParser and set skipLines. DelimParser header field is set as well.

Parameters

Returns:

guessSkipLines

try to figure out how many lines to skip by looking for the line where the number of fields becomes stable.

Parameters

Returns:

isHeader

returns true if the line is a header or comment.

Parameters

Returns:

isIso8601Time

quick-n-dirty check to see if a string appears to be an ISO8601 time. minimally 2000-002T00:00, but also 2000-01-01T00:00:00Z etc. Note that an external code may explicitly indicate that the field is a time, This is just to catch things that are obviously times.

Parameters

Returns:

isKeepFileHeader

Getter for property keepHeader.

Returns:

isRichHeader

return true if the header appears to contain JSON code which could be interpreted as a "Rich Header" (a.k.a. JSONHeadedASCII). This is a very simple test, simply looking for #{ and #} with a colon contained within.

Parameters

Returns:

See Also:

newParser

creates a parser with @param fieldCount fields, named "field0,...,fieldN"

Parameters

Returns:

readFile

Parse the file using the current settings.

Parameters

Returns:

readFirstParseableRecord

returns the first record that the record parser parses successfully. The recordParser should be set and configured enough to identify the fields. If no records can be parsed, then null is returned. The first record should be in the first 1000 lines.

Parameters

Returns:

readFirstRecord

return the first record that the parser would parse. If skipLines is more than the total number of lines, or all lines are comments, then null is returned.

Parameters

Returns:

readStream

Parse the stream using the current settings.

Parameters

Returns:

setCommentPrefix

Records starting with this are not processed as data, for example "#". This is initially "#". Setting this to null disables this check.

Parameters

Returns:

setDelimParser

The DelimParser splits each record into fields using a delimiter like "," or "\\s+".

Parameters

Returns:

setFieldParser

set the special parser for a field.

Parameters

Returns:

setFillValue

numbers that parse to this value are considered to be fill.

Parameters

Returns:

setFixedColumnsParser

looks at the first line after skipping, and splits it to calculate where the columns are. The FixedColumnsParser is the fastest of the three parsers.

Parameters

Returns:

setHeaderDelimiter

set the delimiter which explicitly separates header from the data. For example "-------" could be used. Normally the parser just looks at the number of fields and this is sufficient.

Parameters

Returns:

setKeepFileHeader

Setter for property keepHeader. By default false but if true, the file header ignored by skipLines is put into the property PROPERTY_FILE_HEADER.

Parameters

Returns:

setPropertyPattern

specify the Pattern used to recognize properties. Note property values are not parsed, they are provided as Strings. This is a regular expression with two groups for the property name and value. For example, (.+)=(.+)

Parameters

Returns:

setRecordCountLimit

limit the number of records read. parsing will stop once this number of records is read. This is Integer.MAX_VALUE by default.

Parameters

Returns:

setRecordParser

Setter for property recordParser.

Parameters

Returns:

setRegexParser

The regex parser is a slow parser, but gives precise control.

Parameters

Returns:

setSkipLines

skip a number of lines before trying to parse anything. This can be set to point at the first valid line, and the RecordParser will be configured using that line.

Parameters

Returns:

setValidMax

set the maximum value for any field. Values above this are to be considered invalid.

Parameters

Returns:

setValidMin

set the minimum valid value for any field. Values less than this are to be considered invalid.

Parameters

Returns:

setWhereConstraint

allow constraint for where condition is true. This doesn't need the data to be interpreted for "eq", string equality is checked for nominal data. Note sval is compared after trimming outside spaces.

Parameters

Returns: