foliolib.folio.api.metaStorage.Metastorage

class foliolib.folio.api.metaStorage.Metastorage(tenant: str, okapi: Optional[foliolib.okapi.okapiClient.OkapiClient] = None)

Bases: foliolib.folio.FolioApi

Meta Storage

Parameters

tenant (str) – Tenant id
okapi (OkapiClient, optional) – Instance of OkapiClient. Defaults to None.

Methods

`deletecodemodule`(id_)	Delete code module by id
`deleteconfigmatchkey`(id_)	Delete match key configuration
`deleteglobalrecords`(**kwargs)	Delete global records.
`deleteoaiconfig`()	Update OAI configuration.
`deleteoaipmhclient`(id_)	Delete OAI-PMH client
`deletesource`(id_)	Delete source.
`getcluster`(clusterId)	Get cluster by identifier
`getclusters`()	Get clusters with matchkeyid.
`getcodemodule`(id_)	Retrieve a code module by id
`getcodemodules`(**kwargs)	Retrieve all code modules
`getcollectionoaipmhclient`()	Get all OAI PMH client jobs
`getconfigmatchkey`(id_)	Get match key configuration
`getconfigmatchkeys`(**kwargs)	Get match key configurations
`getglobalrecord`(globalId)	Get record with global identifier.
`getglobalrecords`(**kwargs)	Get records that satisfy CQL query with fields localId, sourceId, globalId.
`getoaiconfig`()	Get OAI configuration
`getoaipmhclient`(id_)	Get OAI-PMH client
`getsource`(id_)	Get source.
`getsources`(**kwargs)	Get sources.
`ingestjobfinish`(id_, **kwargs)	Finish ingest job with either rollback of commit.
`ingestjobinfo`(id_)	Get ingest job information.
`ingestjobrecord`(ingestRecordChunk, id_)	Put records for job.
`initializematchkey`(id_)	Recalculate match key across all records.
`oaiservice`()	OAI service
`postcodemodule`(codeModule)	Create a new code module
`postconfigmatchkey`(matchKey)	Create match key
`postingestjob`(ingestJobRequest)	Create ingest job
`postoaipmhclient`(oai_pmh_client)	Create OAI PMH client job
`postsource`(source)	Create source.
`putcodemodule`(codeModule, id_)	Update code module by id
`putconfigmatchkey`(matchKey, id_)	Update match key configuration.
`putglobalrecords`(ingestRecordRequest)	Create or update records.
`putoaiconfig`(oaiConfig)	Update OAI configuration.
`putoaipmhclient`(oai_pmh_client, id_)	Update OAI-PMH client
`reloadcodemodule`(id_)	Force module to be reloaded
`startoaipmhclient`(id_)	Start OAI PMH client job
`statsmatchkey`(id_)	Get statistics for match key configuration
`statusoaipmhclient`(id_)	Get OAI PMH client status
`stopoaipmhclient`(id_)	Stop OAI PMH client job

deletecodemodule(id_)

Delete code module by id

DELETE /meta-storage/config/modules/{id}

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

deleteconfigmatchkey(id_)

Delete match key configuration

DELETE /meta-storage/config/matchkeys/{id}

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

deleteglobalrecords(**kwargs)

Delete global records.

DELETE /meta-storage/records

Keyword Arguments

query (str) – CQL query

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

deleteoaiconfig()

Update OAI configuration.

DELETE /meta-storage/config/oai

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

deleteoaipmhclient(id_)

Delete OAI-PMH client

DELETE /meta-storage/pmh-clients/{id}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "OAI PMH client configuration",
    "type": "object",
    "properties": {
        "from": {
            "type": "string",
            "description": "OAI PMH from date"
        },
        "id": {
            "type": "string",
            "description": "OAI PMH Client identifier"
        },
        "headers": {
            "type": "object",
            "description": "Additional HTTP headers to be used against OAI server with keys and string values"
        },
        "metadataPrefix": {
            "type": "string",
            "description": "OAI PMH metadata prefix"
        },
        "params": {
            "type": "object",
            "description": "Additional query parameters with keys and string values"
        },
        "resumptionToken": {
            "type": "string",
            "description": "OAI PMH resumption token"
        },
        "numberRetries": {
            "type": "integer",
            "description": "Number of retries for temporary errors before giving up (default 3)"
        },
        "waitRetries": {
            "type": "integer",
            "description": "Number of seconds to wait between retries (default 10)"
        },
        "set": {
            "type": "string",
            "description": "OAI PMH set as returned in verb=ListSets"
        },
        "sourceId": {
            "type": "string",
            "description": "Source identifier used for storing records"
        },
        "sourceVersion": {
            "type": "integer",
            "description": "Source version used for storing records"
        },
        "until": {
            "type": "string",
            "description": "OAI PMH until date"
        },
        "url": {
            "type": "string",
            "description": "OAI PMH server prefix so that PREFIX?verb=.. works"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "sourceId"
    ]
}

deletesource(id_)

Delete source.

DELETE /meta-storage/config/sources/{id}

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

getcluster(clusterId)

Get cluster by identifier

GET /meta-storage/clusters/{clusterId}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "cluster",
    "type": "object",
    "properties": {
        "clusterId": {
            "type": "string",
            "description": "Cluster identifier",
            "format": "uuid"
        },
        "datestamp": {
            "type": "string",
            "description": "date time of modification",
            "format": "date"
        },
        "records": {
            "description": "List of records",
            "type": "array",
            "items": {
                "type": "object",
                "$ref": "globalRecord.json"
            }
        },
        "matchKeys": {
            "description": "Match keys values",
            "type": "array",
            "items": {
                "type": "string"
            }
        }
    },
    "additionalProperties": false,
    "required": [
        "clusterId",
        "records"
    ]
}

getclusters()

Get clusters with matchkeyid. CQL query with matchValue, clusterId fields

GET /meta-storage/clusters

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "cluster records",
    "type": "object",
    "properties": {
        "items": {
            "description": "List of clusters",
            "type": "array",
            "items": {
                "type": "object",
                "$ref": "cluster.json"
            }
        },
        "resultInfo": {
            "description": "Common result set information",
            "type": "object",
            "$ref": "resultInfo.json"
        }
    },
    "additionalProperties": false,
    "required": [
        "items"
    ]
}

getcodemodule(id_)

Retrieve a code module by id

GET /meta-storage/config/modules/{id}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "MetaStorage code module (extensibility)",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "local module identifier"
        },
        "url": {
            "type": "string",
            "description": "HTTP URL of the module (e.g GitHub raw/usercontent)"
        },
        "function": {
            "type": "string",
            "description": "name of the exported function to be invoked, with JSONObjectString -> JSONObjectString signature"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "function"
    ]
}

getcodemodules(**kwargs)

Retrieve all code modules

GET /meta-storage/config/modules

Keyword Arguments

count (str) – control of counting in queries (default: none, enum: [‘exact’, ‘none’])
limit (int) – Limit the number of elements returned in the response (default: 10, minimum: 0)
query (str) – CQL query
offset (int) – Skip over number of elements (default is first element) (default: 0, minimum: 0)

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "List of MetaStorage code modules",
    "type": "object",
    "properties": {
        "modules": {
            "description": "List of code modules",
            "type": "array",
            "items": {
                "type": "object",
                "$ref": "codeModule.json"
            }
        },
        "resultInfo": {
            "description": "Common result set information",
            "type": "object",
            "$ref": "resultInfo.json"
        }
    },
    "additionalProperties": false,
    "required": [
        "modules"
    ]
}

getcollectionoaipmhclient()

Get all OAI PMH client jobs

GET /meta-storage/pmh-clients

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "OAI PMH client configuration",
    "type": "object",
    "properties": {
        "from": {
            "type": "string",
            "description": "OAI PMH from date"
        },
        "id": {
            "type": "string",
            "description": "OAI PMH Client identifier"
        },
        "headers": {
            "type": "object",
            "description": "Additional HTTP headers to be used against OAI server with keys and string values"
        },
        "metadataPrefix": {
            "type": "string",
            "description": "OAI PMH metadata prefix"
        },
        "params": {
            "type": "object",
            "description": "Additional query parameters with keys and string values"
        },
        "resumptionToken": {
            "type": "string",
            "description": "OAI PMH resumption token"
        },
        "numberRetries": {
            "type": "integer",
            "description": "Number of retries for temporary errors before giving up (default 3)"
        },
        "waitRetries": {
            "type": "integer",
            "description": "Number of seconds to wait between retries (default 10)"
        },
        "set": {
            "type": "string",
            "description": "OAI PMH set as returned in verb=ListSets"
        },
        "sourceId": {
            "type": "string",
            "description": "Source identifier used for storing records"
        },
        "sourceVersion": {
            "type": "integer",
            "description": "Source version used for storing records"
        },
        "until": {
            "type": "string",
            "description": "OAI PMH until date"
        },
        "url": {
            "type": "string",
            "description": "OAI PMH server prefix so that PREFIX?verb=.. works"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "sourceId"
    ]
}

getconfigmatchkey(id_)

Get match key configuration

GET /meta-storage/config/matchkeys/{id}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "shared index match key",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "match key identifier"
        },
        "method": {
            "type": "string",
            "description": "method"
        },
        "params": {
            "type": "object",
            "description": "parameter references"
        },
        "update": {
            "type": "string",
            "enum": [
                "ingest",
                "manual"
            ],
            "description": "ingest: update when records are ingested, manual: update only when initialize is called"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "method",
        "params"
    ]
}

getconfigmatchkeys(**kwargs)

Get match key configurations

GET /meta-storage/config/matchkeys

Keyword Arguments

count (str) – control of counting in queries (default: none, enum: [‘exact’, ‘none’])
limit (int) – Limit the number of elements returned in the response (default: 10, minimum: 0)
query (str) – CQL query
offset (int) – Skip over number of elements (default is first element) (default: 0, minimum: 0)

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "match keys",
    "type": "object",
    "properties": {
        "matchKeys": {
            "description": "List of match keys",
            "type": "array",
            "items": {
                "type": "object",
                "$ref": "matchKey.json"
            }
        },
        "resultInfo": {
            "description": "Common result set information",
            "type": "object",
            "$ref": "resultInfo.json"
        }
    },
    "additionalProperties": false,
    "required": [
        "matchKeys"
    ]
}

getglobalrecord(globalId)

Get record with global identifier.

GET /meta-storage/records/{globalId}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "Global record",
    "type": "object",
    "properties": {
        "globalId": {
            "type": "string",
            "description": "Global identifier",
            "format": "uuid"
        },
        "localId": {
            "type": "string",
            "description": "Local identifier"
        },
        "sourceId": {
            "type": "string",
            "description": "Source identifier"
        },
        "payload": {
            "type": "object",
            "description": "Payload in JSON"
        }
    },
    "additionalProperties": false,
    "required": [
        "globalId",
        "localId",
        "sourceId"
    ]
}

getglobalrecords(**kwargs)

Get records that satisfy CQL query with fields localId, sourceId, globalId.

GET /meta-storage/records

Keyword Arguments

count (str) – control of counting in queries (default: none, enum: [‘exact’, ‘none’])
limit (int) – Limit the number of elements returned in the response (default: 10, minimum: 0)
query (str) – CQL query
offset (int) – Skip over number of elements (default is first element) (default: 0, minimum: 0)

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "Global records",
    "type": "object",
    "properties": {
        "items": {
            "description": "List of records",
            "type": "array",
            "items": {
                "type": "object",
                "$ref": "globalRecord.json"
            }
        },
        "resultInfo": {
            "description": "Common result set information",
            "type": "object",
            "$ref": "resultInfo.json"
        }
    },
    "additionalProperties": false,
    "required": [
        "items"
    ]
}

getoaiconfig()

Get OAI configuration

GET /meta-storage/config/oai

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "OAI configuration",
    "type": "object",
    "properties": {
        "transformer": {
            "type": "string",
            "description": "OAI server response transformer module ID"
        },
        "repositoryName": {
            "type": "string",
            "description": "OAI repository name"
        },
        "baseURL": {
            "type": "string",
            "description": "OAI repository base URL"
        },
        "adminEmail": {
            "type": "string",
            "description": "OAI repository admin email"
        }
    },
    "additionalProperties": false
}

getoaipmhclient(id_)

Get OAI-PMH client

GET /meta-storage/pmh-clients/{id}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "OAI PMH client configuration",
    "type": "object",
    "properties": {
        "from": {
            "type": "string",
            "description": "OAI PMH from date"
        },
        "id": {
            "type": "string",
            "description": "OAI PMH Client identifier"
        },
        "headers": {
            "type": "object",
            "description": "Additional HTTP headers to be used against OAI server with keys and string values"
        },
        "metadataPrefix": {
            "type": "string",
            "description": "OAI PMH metadata prefix"
        },
        "params": {
            "type": "object",
            "description": "Additional query parameters with keys and string values"
        },
        "resumptionToken": {
            "type": "string",
            "description": "OAI PMH resumption token"
        },
        "numberRetries": {
            "type": "integer",
            "description": "Number of retries for temporary errors before giving up (default 3)"
        },
        "waitRetries": {
            "type": "integer",
            "description": "Number of seconds to wait between retries (default 10)"
        },
        "set": {
            "type": "string",
            "description": "OAI PMH set as returned in verb=ListSets"
        },
        "sourceId": {
            "type": "string",
            "description": "Source identifier used for storing records"
        },
        "sourceVersion": {
            "type": "integer",
            "description": "Source version used for storing records"
        },
        "until": {
            "type": "string",
            "description": "OAI PMH until date"
        },
        "url": {
            "type": "string",
            "description": "OAI PMH server prefix so that PREFIX?verb=.. works"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "sourceId"
    ]
}

getsource(id_)

Get source.

GET /meta-storage/config/sources/{id}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "shared index match key",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "match key identifier"
        },
        "method": {
            "type": "string",
            "description": "method"
        },
        "params": {
            "type": "object",
            "description": "parameter references"
        },
        "update": {
            "type": "string",
            "enum": [
                "ingest",
                "manual"
            ],
            "description": "ingest: update when records are ingested, manual: update only when initialize is called"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "method",
        "params"
    ]
}

getsources(**kwargs)

Get sources.

GET /meta-storage/sources

Keyword Arguments

limit (int) – Limit the number of elements returned in the response (default: 10, minimum: 0)
query (str) – CQL query
offset (int) – Skip over number of elements (default is first element) (default: 0, minimum: 0)

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "Sources",
    "type": "object",
    "properties": {
        "sources": {
            "description": "List of sources",
            "type": "array",
            "items": {
                "type": "object",
                "$ref": "source.json"
            }
        },
        "resultInfo": {
            "description": "Common result set information",
            "type": "object",
            "$ref": "resultInfo.json"
        }
    },
    "additionalProperties": false,
    "required": [
        "sources"
    ]
}

ingestjobfinish(id_, **kwargs)

Finish ingest job with either rollback of commit.

DELETE /meta-storage/ingest-jobs/{id}

Keyword Arguments

commit (bool) – whether to commit (default: False)

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

ingestjobinfo(id_)

Get ingest job information.

GET /meta-storage/ingest-jobs/{id}

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "Ingest job info",
    "type": "object",
    "properties": {
        "autoRollBackTimeout": {
            "type": "integer",
            "description": "Timeout in seconds"
        }
    },
    "additionalProperties": false
}

ingestjobrecord(ingestRecordChunk, id_)

Put records for job.

PUT /meta-storage/ingest-jobs/{id}

Parameters

ingestRecordChunk (dict) – See Schema below.

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "Ingest job records chunk",
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "localId": {
                "type": "string",
                "description": "Local identifier"
            },
            "delete": {
                "type": "boolean",
                "description": "If true, record is deleted. Default: false"
            },
            "payload": {
                "type": "object",
                "description": "Payload in JSON"
            }
        },
        "additionalProperties": false,
        "required": [
            "localId"
        ]
    }
}

{
    "description": "Ingest job records response",
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "globalId": {
                "type": "string",
                "description": "Global identifier",
                "format": "uuid"
            },
            "matchKeys": {
                "description": "Match keys for these records. Normally not provided for performance reasons, but client can ask for them",
                "type": "object"
            }
        },
        "additionalProperties": false,
        "required": [
            "globalId"
        ]
    }
}

initializematchkey(id_)

Recalculate match key across all records.

PUT /meta-storage/config/matchkeys/{id}/initialize

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "match key statistics",
    "type": "object",
    "properties": {
        "totalRecords": {
            "type": "integer",
            "description": "total records"
        },
        "clustersTotal": {
            "type": "integer",
            "description": "number of clusters"
        },
        "matchValuesPerCluster": {
            "type": "object",
            "description": "key is number of match vales in cluster; value is number of clusters"
        },
        "recordsPerCluster": {
            "type": "object",
            "description": "key is number of records in a cluster; value is number of clusters of this size (key)"
        },
        "recordsPerClusterSample": {
            "type": "object",
            "description": "key is number of records in a cluster; value is array of some cluster identifiers with this size"
        }
    },
    "additionalProperties": true,
    "required": [
        "totalRecords"
    ]
}

oaiservice()

OAI service

GET /meta-storage/oai

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

postcodemodule(codeModule)

Create a new code module

POST /meta-storage/config/modules

Parameters

codeModule (dict) – See Schema below.

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "MetaStorage code module (extensibility)",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "local module identifier"
        },
        "url": {
            "type": "string",
            "description": "HTTP URL of the module (e.g GitHub raw/usercontent)"
        },
        "function": {
            "type": "string",
            "description": "name of the exported function to be invoked, with JSONObjectString -> JSONObjectString signature"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "function"
    ]
}

postconfigmatchkey(matchKey)

Create match key

POST /meta-storage/config/matchkeys

Parameters

matchKey (dict) – See Schema below.

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "shared index match key",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "match key identifier"
        },
        "method": {
            "type": "string",
            "description": "method"
        },
        "params": {
            "type": "object",
            "description": "parameter references"
        },
        "update": {
            "type": "string",
            "enum": [
                "ingest",
                "manual"
            ],
            "description": "ingest: update when records are ingested, manual: update only when initialize is called"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "method",
        "params"
    ]
}

postingestjob(ingestJobRequest)

Create ingest job

POST /meta-storage/ingest-jobs

Parameters

ingestJobRequest (dict) – See Schema below.

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "Ingest job request",
    "type": "object",
    "properties": {
        "sourceId": {
            "type": "string",
            "description": "Source identifier"
        },
        "autoRollBackTimeout": {
            "type": "integer",
            "description": "Timeout in seconds"
        }
    },
    "additionalProperties": false,
    "required": [
        "sourceId"
    ]
}

postoaipmhclient(oai_pmh_client)

Create OAI PMH client job

POST /meta-storage/pmh-clients

Parameters

oai-pmh-client (dict) – See Schema below.

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "OAI PMH client configuration",
    "type": "object",
    "properties": {
        "from": {
            "type": "string",
            "description": "OAI PMH from date"
        },
        "id": {
            "type": "string",
            "description": "OAI PMH Client identifier"
        },
        "headers": {
            "type": "object",
            "description": "Additional HTTP headers to be used against OAI server with keys and string values"
        },
        "metadataPrefix": {
            "type": "string",
            "description": "OAI PMH metadata prefix"
        },
        "params": {
            "type": "object",
            "description": "Additional query parameters with keys and string values"
        },
        "resumptionToken": {
            "type": "string",
            "description": "OAI PMH resumption token"
        },
        "numberRetries": {
            "type": "integer",
            "description": "Number of retries for temporary errors before giving up (default 3)"
        },
        "waitRetries": {
            "type": "integer",
            "description": "Number of seconds to wait between retries (default 10)"
        },
        "set": {
            "type": "string",
            "description": "OAI PMH set as returned in verb=ListSets"
        },
        "sourceId": {
            "type": "string",
            "description": "Source identifier used for storing records"
        },
        "sourceVersion": {
            "type": "integer",
            "description": "Source version used for storing records"
        },
        "until": {
            "type": "string",
            "description": "OAI PMH until date"
        },
        "url": {
            "type": "string",
            "description": "OAI PMH server prefix so that PREFIX?verb=.. works"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "sourceId"
    ]
}

postsource(source)

Create source.

POST /meta-storage/sources

Parameters

source (dict) – See Schema below.

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "Library Source",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "source identifier",
            "format": "uuid"
        },
        "name": {
            "type": "string",
            "description": "method"
        }
    },
    "additionalProperties": false,
    "required": [
        "id"
    ]
}

putcodemodule(codeModule, id_)

Update code module by id

PUT /meta-storage/config/modules/{id}

Parameters

codeModule (dict) – See Schema below.

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "MetaStorage code module (extensibility)",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "local module identifier"
        },
        "url": {
            "type": "string",
            "description": "HTTP URL of the module (e.g GitHub raw/usercontent)"
        },
        "function": {
            "type": "string",
            "description": "name of the exported function to be invoked, with JSONObjectString -> JSONObjectString signature"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "function"
    ]
}

putconfigmatchkey(matchKey, id_)

Update match key configuration.

PUT /meta-storage/config/matchkeys/{id}

Parameters

matchKey (dict) – See Schema below.

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "shared index match key",
    "type": "object",
    "properties": {
        "id": {
            "type": "string",
            "description": "match key identifier"
        },
        "method": {
            "type": "string",
            "description": "method"
        },
        "params": {
            "type": "object",
            "description": "parameter references"
        },
        "update": {
            "type": "string",
            "enum": [
                "ingest",
                "manual"
            ],
            "description": "ingest: update when records are ingested, manual: update only when initialize is called"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "method",
        "params"
    ]
}

putglobalrecords(ingestRecordRequest)

Create or update records.

PUT /meta-storage/records

Parameters

ingestRecordRequest (dict) – See Schema below.

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiFatalError – Internal error

Schema

{
    "description": "Ingest job records request",
    "type": "object",
    "properties": {
        "sourceId": {
            "type": "string",
            "description": "Source identifier"
        },
        "sourceVersion": {
            "type": "integer",
            "description": "Source version"
        },
        "records": {
            "type": "array",
            "description": "Records",
            "$ref": "ingestRecordChunk.json"
        }
    },
    "additionalProperties": false,
    "required": [
        "sourceId",
        "records"
    ]
}

{
    "description": "Ingest job records response",
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "globalId": {
                "type": "string",
                "description": "Global identifier",
                "format": "uuid"
            },
            "matchKeys": {
                "description": "Match keys for these records. Normally not provided for performance reasons, but client can ask for them",
                "type": "object"
            }
        },
        "additionalProperties": false,
        "required": [
            "globalId"
        ]
    }
}

putoaiconfig(oaiConfig)

Update OAI configuration.

PUT /meta-storage/config/oai

Parameters

oaiConfig (dict) – See Schema below.

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "OAI configuration",
    "type": "object",
    "properties": {
        "transformer": {
            "type": "string",
            "description": "OAI server response transformer module ID"
        },
        "repositoryName": {
            "type": "string",
            "description": "OAI repository name"
        },
        "baseURL": {
            "type": "string",
            "description": "OAI repository base URL"
        },
        "adminEmail": {
            "type": "string",
            "description": "OAI repository admin email"
        }
    },
    "additionalProperties": false
}

putoaipmhclient(oai_pmh_client, id_)

Update OAI-PMH client

PUT /meta-storage/pmh-clients/{id}

Parameters

oai-pmh-client (dict) – See Schema below.

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "OAI PMH client configuration",
    "type": "object",
    "properties": {
        "from": {
            "type": "string",
            "description": "OAI PMH from date"
        },
        "id": {
            "type": "string",
            "description": "OAI PMH Client identifier"
        },
        "headers": {
            "type": "object",
            "description": "Additional HTTP headers to be used against OAI server with keys and string values"
        },
        "metadataPrefix": {
            "type": "string",
            "description": "OAI PMH metadata prefix"
        },
        "params": {
            "type": "object",
            "description": "Additional query parameters with keys and string values"
        },
        "resumptionToken": {
            "type": "string",
            "description": "OAI PMH resumption token"
        },
        "numberRetries": {
            "type": "integer",
            "description": "Number of retries for temporary errors before giving up (default 3)"
        },
        "waitRetries": {
            "type": "integer",
            "description": "Number of seconds to wait between retries (default 10)"
        },
        "set": {
            "type": "string",
            "description": "OAI PMH set as returned in verb=ListSets"
        },
        "sourceId": {
            "type": "string",
            "description": "Source identifier used for storing records"
        },
        "sourceVersion": {
            "type": "integer",
            "description": "Source version used for storing records"
        },
        "until": {
            "type": "string",
            "description": "OAI PMH until date"
        },
        "url": {
            "type": "string",
            "description": "OAI PMH server prefix so that PREFIX?verb=.. works"
        }
    },
    "additionalProperties": false,
    "required": [
        "id",
        "url",
        "sourceId"
    ]
}

reloadcodemodule(id_)

Force module to be reloaded

PUT /meta-storage/config/modules/{id}/reload

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

startoaipmhclient(id_)

Start OAI PMH client job

POST /meta-storage/pmh-clients/{id}/start

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

statsmatchkey(id_)

Get statistics for match key configuration

GET /meta-storage/config/matchkeys/{id}/stats

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "match key statistics",
    "type": "object",
    "properties": {
        "totalRecords": {
            "type": "integer",
            "description": "total records"
        },
        "clustersTotal": {
            "type": "integer",
            "description": "number of clusters"
        },
        "matchValuesPerCluster": {
            "type": "object",
            "description": "key is number of match vales in cluster; value is number of clusters"
        },
        "recordsPerCluster": {
            "type": "object",
            "description": "key is number of records in a cluster; value is number of clusters of this size (key)"
        },
        "recordsPerClusterSample": {
            "type": "object",
            "description": "key is number of records in a cluster; value is array of some cluster identifiers with this size"
        }
    },
    "additionalProperties": true,
    "required": [
        "totalRecords"
    ]
}

statusoaipmhclient(id_)

Get OAI PMH client status

GET /meta-storage/pmh-clients/{id}/status

Returns

See Schema below.

Return type

dict

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error

Schema

{
    "description": "OAI PMH clients status",
    "type": "object",
    "properties": {
        "items": {
            "description": "Status each PMH client",
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "config": {
                        "description": "configuration",
                        "type": "object",
                        "$ref": "oai-pmh-client.json"
                    },
                    "error": {
                        "type": "string",
                        "description": "Error message for last error"
                    },
                    "lastActiveTimestamp": {
                        "description": "timestamp when the job was last active (last successful ingest)",
                        "type": "string",
                        "format": "date-time"
                    },
                    "lastRecsPerSec": {
                        "description": "Average records per second for the last run",
                        "type": "integer"
                    },
                    "lastRunningTime": {
                        "description": "Last running time, since the job was (re)started",
                        "type": "string"
                    },
                    "lastTotalRecords": {
                        "description": "Records retrieved during last run",
                        "type": "integer"
                    },
                    "lastStartedTimestamp": {
                        "description": "Timestamp when the job was last started",
                        "type": "string",
                        "format": "date-time"
                    },
                    "status": {
                        "description": "OAI PMH job status",
                        "type": "string",
                        "enum": [
                            "idle",
                            "running"
                        ]
                    },
                    "totalRecords": {
                        "description": "Total number of records fetched",
                        "type": "integer"
                    },
                    "totalDeleted": {
                        "description": "Total number of deleted records",
                        "type": "integer"
                    },
                    "totalInserted": {
                        "description": "Total number of inserted records",
                        "type": "integer"
                    },
                    "totalUpdated": {
                        "description": "Total number of updated records",
                        "type": "integer"
                    },
                    "totalRequests": {
                        "description": "Total HTTP requests",
                        "type": "integer"
                    }
                },
                "additionalProperties": false,
                "required": [
                    "status",
                    "config",
                    "totalRecords",
                    "totalRequests"
                ]
            }
        }
    }
}

stopoaipmhclient(id_)

Stop OAI PMH client job

POST /meta-storage/pmh-clients/{id}/stop

Raises

OkapiRequestError – Bad request
OkapiRequestNotFound – Not Found
OkapiFatalError – Internal error