Files

Files, special table

The Files table contains the list of all the files that have been captured by the read statements in the Envision script. It has a primary dimension named file : ordinal.

This table is intended to support the design data integrity checks. For example, to test files against conditions related to their expected size; or to pinpoint the origin file of an inconsistent line.

show table "My Files" with
  Files.Path
  Files.Age
  Files.ModifiedDate
  Files.ModifiedHour
  Files.ModifiedMinute
  Files.Alias
  Files.Bytes
  Files.Success
  Files.RawLines
  Files.BadLines
  Files.BadDates
  Files.BadNumbers
  Files.MissingValues

The fields are defined as follow:

The table Files is upstream of all the tables obtained through read statements. The following script illustrates this capability:

read "/sample/Orders*.tsv" as Orders with
  Quantity : number

Orders.Path = Files.Path // broadcast

where Orders.Quantity < 0
  show table "Files with negative order quantities" with
      Orders.Path
      group by Orders.Path