org.eclipse.january.dataset

## Class LazyDataset

- java.lang.Object
- - org.eclipse.january.dataset.LazyDatasetBase
  - - org.eclipse.january.dataset.LazyDataset

- All Implemented Interfaces:
    Serializable, Cloneable, ILazyDataset, IMetadataProvider, INameable
  Direct Known Subclasses:
    LazyDynamicDataset

      public class LazyDataset
      extends LazyDatasetBase
      implements Serializable, Cloneable

  See Also:
    Serialized Form

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected LazyDataset` | `base`
    `protected int[]` | `begSlice`
    `protected int[]` | `delSlice`
    `protected int` | `dtype`
    `protected int` | `isize`
    `protected ILazyLoader` | `loader`
    `protected int[]` | `map`
    `protected Map<Class<? extends MetadataType>,List<MetadataType>>` | `oMetadata`
    `protected int[]` | `oShape`
    `protected int` | `postShape`
    `protected int` | `prepShape`
    `protected long` | `size`

    - ### Fields inherited from class org.eclipse.january.dataset.LazyDatasetBase

      `catchExceptions, logger, metadata, name, shape`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `LazyDataset(String name, int dtype, int[] shape, ILazyLoader loader)` Create a lazy dataset
    `LazyDataset(String name, int dtype, int elements, int[] shape, ILazyLoader loader)` Create a lazy dataset

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected SliceND` | `calcTrueSlice(SliceND slice)`
    `LazyDataset` | `clone()` Clone dataset
    `static LazyDataset` | `createLazyDataset(Dataset dataset)` Create a lazy dataset based on in-memory data (handy for testing)
    `protected SliceND` | `createSlice(int[] nstart, int[] nstop, int[] nstep)`
    `boolean` | `equals(Object obj)`
    `int` | `getDType()` Can return -1 for unknown
    `int` | `getElementsPerItem()` Can return -1 for unknown
    `static int` | `getMaxSliceLength(ILazyDataset lazySet, int dimension)` Gets the maximum size of a slice of a dataset in a given dimension which should normally fit in memory.
    `int` | `getSize()` The size of the dataset is the number of items in the array
    `Dataset` | `getSlice(IMonitor monitor, int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `Dataset` | `getSlice(IMonitor monitor, Slice... slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(IMonitor monitor, SliceND slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `Dataset` | `getSlice(Slice... slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(SliceND slice)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(Slice... slice)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(SliceND slice)` Get a slice of the dataset.
    `LazyDataset` | `getTransposedView(int... axes)` Permute copy of dataset's axes so that given order is old order:
    `void` | `setShape(int... shape)` Set a compatible shape for dataset.
    `LazyDataset` | `squeezeEnds()` Remove dimensions of 1 from ends of shape of the dataset
    `String` | `toString()`
    `protected IDataset` | `transformInput(IDataset data)`

    - ### Methods inherited from class org.eclipse.january.dataset.LazyDatasetBase

      `addMetadata, checkPermutatedAxes, clearMetadata, copyMetadata, copyMetadata, createFromSerializable, dirtyMetadata, findMetadataTypeSubInterfaces, getElementClass, getErrorMetadata, getErrors, getFirstMetadata, getMetadata, getMetadata, getName, getRank, getShape, hasErrors, hashCode, reshapeMetadata, restoreMetadata, setDirty, setErrors, setMetadata, setName, sliceMetadata, transposeMetadata`

    - ### Methods inherited from class java.lang.Object

      `finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### oShape

          protected int[] oShape

    - #### size

          protected long size

    - #### dtype

          protected int dtype

    - #### isize

          protected int isize

    - #### loader

          protected ILazyLoader loader

    - #### base

          protected LazyDataset base

    - #### prepShape

          protected int prepShape

    - #### postShape

          protected int postShape

    - #### begSlice

          protected int[] begSlice

    - #### delSlice

          protected int[] delSlice

    - #### map

          protected int[] map

    - #### oMetadata

          protected Map<Class<? extends MetadataType>,List<MetadataType>> oMetadata

  - ### Constructor Detail

    - #### LazyDataset

          public LazyDataset(String name,
                     int dtype,
                     int elements,
                     int[] shape,
                     ILazyLoader loader)

      Create a lazy dataset
      Parameters:
        `name` -
        `dtype` - dataset type
        `elements` -
        `shape` -
        `loader` -

    - #### LazyDataset

          public LazyDataset(String name,
                     int dtype,
                     int[] shape,
                     ILazyLoader loader)

      Create a lazy dataset
      Parameters:
        `name` -
        `dtype` - dataset type
        `shape` -
        `loader` -

  - ### Method Detail

    - #### createLazyDataset

          public static LazyDataset createLazyDataset(Dataset dataset)

      Create a lazy dataset based on in-memory data (handy for testing)
      Parameters:
        `dataset` -

    - #### getDType

          public int getDType()

      Can return -1 for unknown
      Specified by:
        `getDType` in class `LazyDatasetBase`
      Returns:
        type of dataset item

    - #### getElementsPerItem

          public int getElementsPerItem()

      Can return -1 for unknown
      Specified by:
        `getElementsPerItem` in interface `ILazyDataset`
      Returns:
        Number of elements per item

    - #### getSize

          public int getSize()

      Description copied from interface: `ILazyDataset`
      The size of the dataset is the number of items in the array
      Specified by:
        `getSize` in interface `ILazyDataset`
      Returns:
        number of data items

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

    - #### equals

          public boolean equals(Object obj)

      Overrides:
        `equals` in class `LazyDatasetBase`

    - #### clone

          public LazyDataset clone()

      Description copied from interface: `ILazyDataset`
      Clone dataset
      Specified by:
        `clone` in interface `ILazyDataset`
      Overrides:
        `clone` in class `LazyDatasetBase`
      Returns:
        a (shallow) copy of dataset

    - #### setShape

          public void setShape(int... shape)

      Description copied from interface: `ILazyDataset`
      Set a compatible shape for dataset. A shape is compatible if it has the capacity to contain the same number of items
      Specified by:
        `setShape` in interface `ILazyDataset`

    - #### squeezeEnds

          public LazyDataset squeezeEnds()

      Description copied from interface: `ILazyDataset`
      Remove dimensions of 1 from ends of shape of the dataset
      Specified by:
        `squeezeEnds` in interface `ILazyDataset`

    - #### getSlice

          public Dataset getSlice(int[] start,
                         int[] stop,
                         int[] step)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(Slice... slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(SliceND slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `slice` - an n-D slice
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                         Slice... slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSliceView

          public LazyDataset getSliceView(Slice... slice)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The sliced view of a lazy dataset

    - #### getSliceView

          public LazyDataset getSliceView(int[] start,
                                 int[] stop,
                                 int[] step)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The sliced view of a lazy dataset

    - #### getSliceView

          public LazyDataset getSliceView(SliceND slice)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `slice` - an n-D slice
      Returns:
        The sliced view of a lazy dataset

    - #### getTransposedView

          public LazyDataset getTransposedView(int... axes)

      Description copied from interface: `ILazyDataset`
      Permute copy of dataset's axes so that given order is old order:

            axisPerm = (p(0), p(1),...) => newdata(n(0), n(1),...) = olddata(o(0), o(1), ...)
            such that n(i) = o(p(i)) for all i

      I.e. for a 3D dataset (1,0,2) implies the new dataset has its 1st dimension running along the old dataset's 2nd dimension and the new 2nd is the old 1st. The 3rd dimension is left unchanged.
      Specified by:
        `getTransposedView` in interface `ILazyDataset`
      Parameters:
        `axes` - if zero length then axes order reversed
      Returns:
        remapped view of data

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                         int[] start,
                         int[] stop,
                         int[] step)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                         SliceND slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
        `slice` - an n-D slice
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### calcTrueSlice

          protected final SliceND calcTrueSlice(SliceND slice)

    - #### transformInput

          protected final IDataset transformInput(IDataset data)

    - #### createSlice

          protected SliceND createSlice(int[] nstart,
                            int[] nstop,
                            int[] nstep)

    - #### getMaxSliceLength

          public static int getMaxSliceLength(ILazyDataset lazySet,
                              int dimension)

      Gets the maximum size of a slice of a dataset in a given dimension which should normally fit in memory. Note that it might be possible to get more in memory, this is a conservative estimate and seems to almost always work at the size returned; providing Xmx is less than the physical memory. To get more in memory increase -Xmx setting or use an expression which calls a rolling function (like rmean) instead of slicing directly to memory.
      Parameters:
        `lazySet` -
        `dimension` -
      Returns:
        maximum size of dimension that can be sliced.

Copyright © 2014–2017 Eclipse Foundation. All rights reserved.