lc_catalog

LcCatalog

Bases: StaticCatalog

A catalog that supports adding and manipulating resources during runtime

__del__()

This is unreliable- temp directories tend to accumulate

Returns:

Type	Description

add_existing_archive(archive, interfaces=None, store=True, **kwargs)

Makes a resource record out of an existing archive. by default, saves it in the catalog's resource dir

Parameters:

Name	Type	Description	Default
archive			required
interfaces			None
store		[True] if False, don't save the record - use it for this session only	True
kwargs			{}

Returns:

Type	Description

add_resource(resource, store=True, replace=False)

Add an existing LcResource to the catalog.

Parameters:

Name	Type	Description	Default
resource			required
store		[True] permanently store this resource	True
replace		[False] if the resource already exists, remove it and replace it with the new resource	False

Returns:

Type	Description

blackbook_authenticate(blackbook_url=None, username=None, password=None, token=None, **kwargs)

Opens an authenticated session with the designated blackbook server. Credentials can either be provided to the method as arguments, or if omitted, they can be obtained through a form. If a token is provided, it is used in lieu of a password workflow

Parameters:

Name	Type	Description	Default
blackbook_url			None
username			None
password			None
token			None
kwargs		passed to RestClient. save_credentials=True. verify: provide path to self-signed certificate	{}

Returns:

Type	Description

blackbook_request_third_party_resource(origin, resource_for)

Parameters:

Name	Type	Description	Default
origin		the data resource	required
resource_for		the foreground that needs access	required

Returns:

Type	Description

create_descendant(origin, interface=None, source=None, force=False, signifier=None, strict=True, priority=None, **kwargs)

Parameters:

Name	Type	Description	Default
origin			required
interface			None
source			None
force		overwrite if exists	False
signifier		semantic descriptor for the new descendant (optional)	None
strict			True
priority			None
kwargs			{}

Returns:

Type	Description

create_source_cache(source, static=False)

Creates a cache of the named source's current contents, to speed up access to commonly used entities. source must be either a key present in self.sources, or a name or nickname found in self.names

Parameters:

Name	Type	Description	Default
source			required
static		[False] create archives of a static archive (use to force archival of a complete database)	False

Returns:

Type	Description

delete_resource(resource, delete_source=None, delete_cache=True)

Removes the resource from the resolver and also removes the serialization of the resource. Also deletes the resource's source (as well as all files in the same directory that start with the same name) under the following circumstances: (resource is internal AND resources_with_source(resource.source) is empty AND resource.source is a file) This can be overridden using he delete_source param (see below)

We also need to remove any implementations that use the resource.

Parameters:

Name	Type	Description	Default
resource		an LcResource	required
delete_source		[None] If None, follow default behavior. If True, delete the source even if it is not internal (source will not be deleted if other resources refer to it OR if it is not a file). If False, do not delete the source.	None
delete_cache		[True] whether to delete cache files (you could keep them around if you expect to need them again and you don't think the contents will have changed)	True

Returns:

Type	Description

download_file(url=None, md5sum=None, force=False, localize=True)

Download a file from a remote location into the catalog and return its local path. Optionally validate the download with an MD5 digest.

Parameters:

Name	Type	Description	Default
url			None
md5sum			None
force			False
localize		whether to return the filename relative to the catalog root	True

Returns:

Type	Description
	the full path to the downloaded file

gen_interfaces(origin, itype=None, strict=False)

Override parent method to also create local backgrounds

Parameters:

Name	Type	Description	Default
origin			required
itype			None
strict			False

Returns:

Type	Description

get_blackbook_resources(origin, store=False, assign_ref=None, **kwargs)

Use a blackbook server to obtain resources for a given origin.

Parameters:

Name	Type	Description	Default
origin			required
store		whether to save resources. by default we don't, assuming the tokens are short-lived.	False
assign_ref		give the resource a named ref locally	None
kwargs		init args to add to returned resources, such as 'verify' certificate paths	{}

Returns:

Type	Description

index_ref(origin, interface=None, source=None, priority=60, save=True, force=False, strict=True)

Creates an index for the specified resource. 'origin' and 'interface' must resolve to one or more LcResources that all have the same source specification. That source archive gets indexed, and index resources are created for all the LcResources that were returned.

Performs load_all() on the source archive, writes the archive to a compressed json file in the local index directory, and creates a new LcResource pointing to the JSON file. Aborts if the index file already exists (override with force=True).

Parameters:

Name	Type	Description	Default
origin			required
interface		[None]	None
source		find_single_source input	None
priority		[60] priority setting for the new index -- authentic source is highest	60
save		[True] whether to save the index	True
force		[False] if True, overwrite existing index	False
strict		[True] whether to be strict	True

Returns:

Type	Description

make_tester(**kwargs) classmethod

Sets a flag that tells the rootdir to be deleted when the catalog is garbage collected

Parameters:

Name	Type	Description	Default
kwargs			{}

Returns:

Type	Description

new_resource(reference, source, ds_type, store=True, **kwargs)

Create a new data resource by specifying its properties directly to the constructor

Parameters:

Name	Type	Description	Default
reference			required
source			required
ds_type			required
store		[True] permanently store this resource	True
kwargs		interfaces=None, priority=0, static=False; **kwargs passed to archive constructor	{}

Returns:

Type	Description

purge_resource_archive(resource)

• find all cached queries that could return the resource
• check their cached ifaces to see if they use our archive
• delete those entries from the cache

Parameters:

Name	Type	Description	Default
resource	LcResource		required

Returns:

Type	Description

refresh_xdb_tokens(remote_origin=None)

requires an active blackbook client (try blackbook_authenticate() if it has expired)

Parameters:

Name	Type	Description	Default
remote_origin			None

Returns:

Type	Description