Module Zarr

zarr Provides an Ocaml implementation of the Zarr version 3 storage format specification. It supports creation of arrays and groups as well as chunking arrays along any dimension. One can store a Zarr hierarchy in memory or on disk. Zarr also supports reading zarr hierarchies created using other implementations, as long as they are spec-compliant.

Consult the examples and limitations for more info.

References

Node

module Node : sig ... end

This module provides functionality for manipulating Zarr nodes.

Metadata

module Metadata : sig ... end

This module provides functionality for manipulating a Zarr node's metadata JSON document.

Storage

module Storage : sig ... end
module Memory : sig ... end
module Types : sig ... end

Codecs

module Codecs : sig ... end

An array has an associated list of codecs. Each codec specifies a bidirectional transform (an encode transform and a decode transform). This module contains building blocks for creating and working with a chain of codecs.

Indexing

module Indexing = Ndarray.Indexing

Utils

module Util : sig ... end

Ndarray

module Ndarray : sig ... end

Examples

Create, read & write array.

Here we show how the library's asynchronous API using Lwt's concurrency monad can be used.

open Zarr.Metadata
open Zarr.Node
open Zarr.Codecs
open Zarr_lwt.Storage
open FilesystemStore.Deferred.Syntax

let _ =
  Lwt_main.run begin
    let store = FilesystemStore.create "testdata.zarr" in
    let group_node = GroupNode.of_path "/some/group" in
    let* () = FilesystemStore.create_group store group_node in
    let array_node = ArrayNode.(group_node / "name") in
    let* () = FilesystemStore.create_array
      ~codecs:[`Bytes BE] ~shape:[|100; 100; 50|] ~chunks:[|10; 15; 20|]
      Bigarray.Float32 Float.neg_infinity array_node store in
    let slice = Owl_types.[|R [0; 20]; I 10; R []|] in
    let* x = FilesystemStore.read_array store array_node slice Bigarray.Float32 in
    let x' = Owl.Dense.Ndarray.Generic.map (fun _ -> Owl_stats_dist.uniform_rvs 0. 10.) x
    in FilesystemStore.write_array store array_node slice x'
  end

Extension Points

This library also provides custom extensions not defined in the version 3 specification. These are tabulated below:

Extension Point

Details

Data Types

char, complex32, int (63-bit integer), nativeint

Limitations

Although this implementation tries to be spec compliant, it does come with a few limitations: