# Data representation

All designs handled by the OApackage are integer valued. The designs (whether these are orthogonal arrays, optimal designs or conferences designs)
are stored in an `array_link()`

object. The definitions of orthogonal arrays, optimal designs and conference designs are included in the section Definitions of arrays and designs.

## Data structures

The package contains several data structures. Here, we describe the main structures and their use.

`array_link()`

The structure containing an orthogonal array is called the

`array_link()`

structure. Lists of arrays are stored in the`arraylist_t()`

object, which is implemented as a`std::deque`

container.`arrayfile_t()`

This object allows for reading and writing of arrays to disk.

`arraydata_t()`

The structure describing a certain class of orthogonal arrays or optimal designs.

`conference_t()`

The structure describing a certain class of conference designs.

`array_transformation_t()`

The structure describing a transformation of an orthogonal array, which includes row and column permutations, as well as permutations to the symbols in one or more columns.

`conference_transformation_t()`

The structure describing a transformation of conference design or double conference design, which includes row and column permutations, as well as sign switches to the elements in one or more rows and columns.

## Representing arrays

The structure containing an orthogonal array is called the
`array_link`

structure. It consists of a specified number of rows and
columns, the data (integer valued) and an index. In the Python interface, the `array_link()`

object can be indexed just as
normal arrays.

It is also possible to convert to a Numpy array. The
`array_link`

object implements the Python array interface, so most
operations from packages such as Numpy work on the `array_link()`

object.

Array representation and indexing in Python

```
>>> import oapackage; import numpy as np
>>> al=oapackage.exampleArray(0)
>>> al.showarray()
array:
0 0
0 0
0 1
0 1
1 0
1 0
1 1
1 1
>>> al[2,1]
1
>>> X=np.array(al)
>>> X
array([[0, 0],
[0, 0],
[0, 1],
[0, 1],
[1, 0],
[1, 0],
[1, 1],
[1, 1]], dtype=int16)
```

The C++ class is `array_link`

.

## Classes of arrays

The `arraydata_t`

object represents data about a class of orthogonal
arrays, e.g. the class \({\operatorname{OA}(N; t; s^k)}\); see Definitions of arrays and designs. The `conference_t`

object represents data about a class of conference designs.

## Array transformations

Transformations of (orthogonal) arrays consist of row, column and
level permutations. A transformation is represented by
the `array_transformation_t`

object.

For a given transformation, the column permutations are applied first, then the level permutations and finally the row permutations. The level and column permutations are not commutative.

The conference design transformations also allow for row sign switches and are
described by the class `conference_transformation_t`

.

## Reading and writing arrays

Reading and writing arrays to disk can be done with the `arrayfile_t`

class.

Write an array or a list of arrays to disk

```
>>> import oapackage
>>> list_of_arrays = [oapackage.exampleArray(24), oapackage.exampleArray(25)]
>>> _ = oapackage.writearrayfile('test.oa', list_of_arrays)
>>> oapackage.oainfo('test.oa')
file test.oa: 64 rows, 16 columns, 2 arrays, mode text, nbits 0
>>> al=oapackage.exampleArray()
>>> af=oapackage.arrayfile_t('test.oa', al.n_rows, al.n_columns)
>>> af.append_array(al)
>>> print(af)
file test.oa: 8 rows, 2 columns, 1 arrays, mode text, nbits 8
>>> af.closefile()
```

The arrays can be written in text or binary format. For more details on the file format, see the section File formats.

The Python interface is `oalib.arrayfile_t()`

and the C++ interface
is `arrayfile_t`

.

## File formats

The Orthogonal Array package stores arrays in a custom file format. There is a text format which is easily readable by humans and a binary format which is faster to process and memory efficient.

### Plain text array files

Arrays are stored in plain text files with extension `.oa`

. The first line
contains the number of columns, the number of rows and the number of
arrays (or -1 if the number of arrays is not specified). Then, for each
array, a single line with the index of the array, followed by \({N}\) lines
containing the array.

A typical example of a text file is the following:

```
5 8 1
1
0 0 0 0 0
0 0 0 1 1
0 1 1 0 0
0 1 1 1 1
1 0 1 0 1
1 0 1 1 0
1 1 0 0 1
1 1 0 1 0
-1
```

This file contains exactly 1 array with 8 rows and 5 columns.

### Binary array files

Every binary file starts with a header, which has the following format:

```
[INT32] 65 (magic identifier)
[INT32] b: Format: number of bits per number. Currently supported are 1 and 8
[INT32] N: number of rows
[INT32] k: kumber of columns
[INT32] Number of arrays (can be -1 if unknown)
[INT32] Binary format number: 1001: normal, 1002: binary diff, 1003: binary diff zero
[INT32] Reserved integer
[INT32] Reserved integer
```

The format of the remainder of the binary file depends on the binary format specified. For the normal binary format, the format is as follows. For each array, the number is specified in the header:

```
[INT32] Index
[Nxk elements] The elements contain b bits
```

If the number of bits per number is 1 (e.g. a 2-level array), then the
data is padded with zeros to a multiple of 64 bits. The data of the
array is stored in column-major order. The binary file format allows for
random access reading and writing. The `binary diff`

and ```
binary diff
zero
```

formats are special formats.

A binary array file can be compressed using gzip. Most tools in the Orthogonal Array package can read these compressed files transparently. Writing to compressed array files is not supported at the moment.

### Data files

The analysis tool (`oaanalyse`

) writes data to disk in binary format.
The format consists of a binary header:

```
[FLOAT64] Magic number 30397995;
[FLOAT64] Magic number 12224883;
[FLOAT64] nc: Number of rows
[FLOAT64] nr: Number of columns
```

After the header there follow `nc*nr`

`[FLOAT64]`

values.

### MD5 sums

To check data integrity on disk, the packages includes functions to generate MD5 sums of designs.

Calculate md5 sum of a design

```
>>> import oapackage; al=oapackage.exampleArray(0)
>>> al.md5()
'6454c492239a8e01e3c01a864583abf2'
```

The C++ functions are `array_link::md5()`

and `md5()`

.

## Command line interface

Several command line tools are included in the Orthogonal Array package. For each tool,
help can be obtained from the command line by using the switch `-h`

.
The tools include the following:

- oainfo
This program reads Orthogonal Array package data files and reports the contents of the files. For example:

$ oainfo result-8.2-2-2-2.oa Orthogonal Array package 1.8.7 oainfo: reading 1 file(s) file result-8.2-2-2.oa: 8 rows, 3 columns, 2 arrays, mode text, nbits 0 $

- oacat
Shows the contents of a file with orthogonal arrays for a data file.

- oacheck
Checks or reduces an array to canonical form.

- oaextendsingle
Extends a set of arrays in LMC form with one or more columns.

- oacat
Shows the contents of an array file or data file.

Usage:

`oacat [OPTIONS] [FILES]`

- oajoin
Reads one or more files from disk and join all the array files into a single list.

- oasplit
Takes a single array file as input and splits the arrays into a specified number of output files.

- oapareto
Calculates the set of Pareto optimal arrays in a file with arrays.

- oaanalyse
Calculates various statistical properties of arrays in a file. The properties are described in section Properties of designs.