annotationengine

AnnotationClientV2(server_address, auth_header, api_version, endpoints, server_name, aligned_volume_name, verify=True, max_retries=None, pool_maxsize=None, pool_block=None, over_client=None, schema_client=None)

Bases: ClientBase

create_table(table_name, schema_name, description, voxel_resolution, reference_table=None, track_target_id_updates=None, flat_segmentation_source=None, user_id=None, aligned_volume_name=None, write_permission='PRIVATE', read_permission='PUBLIC', notice_text=None)

Creates a new data table based on an existing schema

Parameters:

Name	Type	Description	Default
table_name	str	Name of the new table. Cannot be the same as an existing table	required
schema_name	str	Name of the schema for the new table.	required
description	str	Human readable description for what is in the table. Should include information about who generated the table What data it covers, and how it should be interpreted. And who should you talk to if you want to use it. An Example: a manual synapse table to detect chandelier synapses on 81 PyC cells with complete AISs [created by Agnes - agnesb@alleninstitute.org, uploaded by Forrest]	required
voxel_resolution	List[float]	voxel resolution points will be uploaded in, typically nm, i.e [1,1,1] means nanometers [4,4,40] would be 4nm, 4nm, 40nm voxels	required
reference_table	str	If the schema you are using is a reference schema Meaning it is an annotation of another annotation. Then you need to specify what the target table those annotations are in.	None
track_target_id_updates	bool	Indicates whether to automatically update reference table's foreign key if target annotation table row is updated.	None
flat_segmentation_source	str	the source to a flat segmentation that corresponds to this table i.e. precomputed:\gs:\mybucket his_tables_annotation	None
user_id	int	If you are uploading this schema on someone else's behalf and you want to link this table with their ID, you can specify it here Otherwise, the table will be created with your userID in the user_id column.	None
aligned_volume_name	str	Name of the aligned_volume. If None, uses the one specified in the client.	None
write_permission	str	What permissions to give the table for writing. One of PRIVATE: only you can write to this table (DEFAULT) GROUP: only members that share a group with you can write (excluding some groups) PUBLIC: Anyone can write to this table. Note all data is logged, and deletes are done by marking rows as deleted, so all data is always recoverable	'PRIVATE'
read_permission	str	What permissions to give the table for reading. One of PRIVATE: only you can read this table. Intended to be used for sorting out bugs. GROUP: only members that share a group with you can read (intended for within group vetting) PUBLIC: anyone with permissions to read this datastack can read this data (DEFAULT)	'PUBLIC'
notice_text	str	Text the user will see when querying this table. Can be used to warn users of flaws, and uncertainty in the data, or to advertise citations that should be used with this table. Defaults to None, no text. If you want to remove text, send empty string.	None

Returns:

Type	Description
json	Response JSON

Examples:

Basic annotation table:

description = "Some description about the table"
voxel_res = [4,4,40]
client.create_table("some_synapse_table", "synapse", description, voxel_res)

delete_annotation(table_name, annotation_ids, aligned_volume_name=None)

Delete one or more annotations in a table. Annotations that are deleted are recorded as 'non-valid' but are not physically removed from the table.

Parameters:

Name	Type	Description	Default
table_name	str	Name of the table where annotations will be added	required
annotation_ids	(dict or list)	A list of (or a single) dict of schematized annotation data matching the target table. each dict must contain an "id" field which is the ID of the annotation to update	required
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
json	Response JSON: a list of new annotation IDs.

delete_table(table_name, aligned_volume_name=None)

Marks a table for deletion requires super admin privileges

Parameters:

Name	Type	Description	Default
table_name	str	name of table to mark for deletion	required
aligned_volume_name	str	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
json	Response JSON

get_annotation(table_name, annotation_ids, aligned_volume_name=None)

Retrieve an annotation or annotations by id(s) and table name.

Parameters:

Name	Type	Description	Default
table_name	str	Name of the table	required
annotation_ids	int or iterable	ID or IDS of the annotation to retreive	required
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
list	Annotation data

get_annotation_count(table_name, aligned_volume_name=None)

Get number of annotations in a table

Parameters:

Name	Type	Description	Default
table_name	str	name of table to mark for deletion	required
aligned_volume_name	str	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
int	number of annotations

get_table_metadata(table_name, aligned_volume_name=None)

Get metadata about a table

Parameters:

Name	Type	Description	Default
table_name	str	name of table to mark for deletion	required
aligned_volume_name	str	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
json	metadata about table

get_tables(aligned_volume_name=None)

Gets a list of table names for a aligned_volume_name

Parameters:

Name	Type	Description	Default
aligned_volume_name	str or None	Name of the aligned_volume, by default None. If None, uses the one specified in the client. Will be set correctly if you are using the framework_client	None

Returns:

Type	Description
list	List of table names

post_annotation(table_name, data, aligned_volume_name=None)

Post one or more new annotations to a table in the AnnotationEngine. All inserted annotations will be marked as 'valid'. To invalidate annotations refer to 'update_annotation', 'update_annotation_df' and 'delete_annotation' methods.

Parameters:

Name	Type	Description	Default
table_name	str	Name of the table where annotations will be added	required
data	(dict or list)	A list of (or a single) dict of schematized annotation data matching the target table.	required
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
json	Response JSON

post_annotation_df(table_name, df, position_columns, aligned_volume_name=None)

Post one or more new annotations to a table in the AnnotationEngine. All inserted annotations will be marked as 'valid'. To invalidate annotations see 'update_annotation', 'update_annotation_df' and 'delete_annotation' methods.

Parameters:

Name	Type	Description	Default
table_name	str	Name of the table where annotations will be added	required
df	DataFrame	A pandas dataframe containing the annotations. Columns should be fields in schema, position columns need to be called out in position_columns argument.	required
position_columns	Optional[Union[Iterable[str], Mapping[str, str]]]	if None, will look for all columns with 'X_position' in the name and assume they go in fields called "X". if Iterable assumes each column given ends in _position. (i.e. ['pt_position'] if 'pt' is the name of the position field in schema) if Mapping, keys are names of columns in dataframe, values are the names of the fields (i.e. {'pt_column': 'pt'} would be correct if you had one column named 'pt_column' which needed to go into a schema with a position column called 'pt')	required
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
json	Response JSON

process_position_columns(df, position_columns) staticmethod

Process a dataframe into a list of dictionaries

Parameters:

Name	Type	Description	Default
df	DataFrame	Dataframe to process	required
position_columns	Optional[Union[Iterable[str], Mapping[str, str]]]	See .post_annotation_df	required

Returns:

Type	Description
dict	Annotations ready for posting

raise_for_status(r, log_warning=True) staticmethod

Raises requests.HTTPError, if one occurred.

stage_annotations(table_name=None, schema_name=None, update=False, id_field=False, table_resolution=None, annotation_resolution=None)

Get a StagedAnnotations object to help produce correctly formatted annotations for a given table or schema. StagedAnnotation objects can be uploaded directly with upload_staged_annotations.

Parameters:

Name	Type	Description	Default
table_name	str	Table name to stage annotations for, by default None.	None
schema_name	str	Schema name to use to make annotations. Only needed if the table_name is not set, by default None	None
update	bool	Set to True if individual annotations are going to be updated, by default False.	False
id_field	bool	Set to True if id fields are to be specified. Not needed if update is True, which always needs id fields. Optional, by default False	False
table_resolution	list - like or None	Voxel resolution of spatial points in the table in nanometers. This is found automatically from the info service if a table name is provided, by default None. If annotation_resolution is also set, this allows points to be scaled correctly for the table.	None
annotation_resolution	list - like	Voxel resolution of spatial points provided by the user when creating annotations. If the table resolution is also available (manually or from the info service), annotations are correctly rescaled for the volume. By default, None.	None

update_annotation(table_name, data, aligned_volume_name=None)

Update one or more new annotations to a table in the AnnotationEngine. Updating is implemented by invalidating the old annotation and inserting a new annotation row, which will receive a new primary key ID.

Notes

If annotations ids were user provided upon insertion the database will autoincrement from the current max id in the table.

Parameters:

Name	Type	Description	Default
table_name	str	Name of the table where annotations will be added	required
data	(dict or list)	A list of (or a single) dict of schematized annotation data matching the target table. each dict must contain an "id" field which is the ID of the annotation to update	required
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
json	Response JSON: a list of new annotation IDs.

update_annotation_df(table_name, df, position_columns, aligned_volume_name=None)

Update one or more annotations to a table in the AnnotationEngine using a dataframe as format. Updating is implemented by invalidating the old annotation and inserting a new annotation row, which will receive a new primary key ID.

Notes

If annotations ids were user provided upon insertion the database will autoincrement from the current max id in the table.

Parameters:

Name	Type	Description	Default
table_name	str	Name of the table where annotations will be added	required
df	DataFrame	A pandas dataframe containing the annotations. Columns should be fields in schema, position columns need to be called out in position_columns argument.	required
position_columns	Iterable[str] or Mapping[str, str] or None	if None, will look for all columns with 'X_position' in the name and assume they go in fields called "X". if Iterable assumes each column given ends in _position. (i.e. ['pt_position'] if 'pt' is the name of the position field in schema) if Mapping, keys are names of columns in dataframe, values are the names of the fields (i.e. {'pt_column': 'pt'} would be correct if you had one column named 'pt_column' which needed to go into a schema with a position column called 'pt')	required
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
json	Response JSON

update_metadata(table_name, description=None, flat_segmentation_source=None, read_permission=None, write_permission=None, user_id=None, notice_text=None, aligned_volume_name=None)

Update the metadata on an existing table

Parameters:

Name	Type	Description	Default
table_name	str		required
description	str	Defaults to None (will not update).	None
flat_segmentation_source	str	Defaults to None (will not update).	None
read_permission	str	What permissions to give the table for reading. One of PRIVATE: only you can read this table. Intended to be used for sorting out bugs. GROUP: only members that share a group with you can read (intended for within group vetting) PUBLIC: anyone with permissions to read this datastack can read this data Defaults to None (will not update).	None
write_permission	str	What permissions to give the table for writing. One of PRIVATE: only you can write to this table GROUP: only members that share a group with you can write (excluding some groups) PUBLIC: Anyone can write to this table. Note all data is logged, and deletes are done by marking rows as deleted, so all data is always recoverable Defaults to None (will not update).	None
user_id	int	Note, if you use this you will not be able to update the metadata on this table any longer and depending on permissions may not be able to read or write to it Defaults to None. (will not update)	None
notice_text	str	Text the user will see when querying this table. Can be used to warn users of flaws, and uncertainty in the data, or to advertise citations that should be used with this table. Defaults to None. (will not update)	None
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

upload_staged_annotations(staged_annos, aligned_volume_name=None)

Upload annotations directly from an Annotation Guide object. This method uses the options specified in the object, including table name and if the annotation is an update or not.

Parameters:

Name	Type	Description	Default
staged_annos	AnnotationGuide	AnnotationGuide object with a specified table name and a collection of annotations already filled in.	required
aligned_volume_name	str or None	Name of the aligned_volume. If None, uses the one specified in the client.	None

Returns:

Type	Description
List or dict	If new annotations are posted, a list of ids. If annotations are being updated, a dictionary with the mapping from old ids to new ids.

AnnotationClient(server_address, dataset_name=None, aligned_volume_name=None, auth_client=None, api_version='latest', verify=True, max_retries=None, pool_maxsize=None, pool_block=None, over_client=None)

Factory for returning AnnotationClient

Parameters:

Name	Type	Description	Default
server_address	str	server_address to use to connect to (i.e. https://minniev1.microns-daf.com)	required
dataset_name	str	Name of the datastack.	None
auth_client	AuthClient or None	Authentication client to use to connect to server. If None, do not use authentication.	None
api_version	str or int (default: latest)	What version of the api to use, 0: Legacy client (i.e www.dynamicannotationframework.com) 2: new api version, (i.e. minniev1.microns-daf.com) 'latest': default to the most recent (current 2)	'latest'
verify	str (default : True)	whether to verify https	True
max_retries	Int or None	Set the number of retries per request, by default None. If None, defaults to requests package default.	None
pool_block	Bool or None	If True, restricts pool of threads to max size, by default None. If None, defaults to requests package default.	None
pool_maxsize	Int or None	Sets the max number of threads in the pool, by default None. If None, defaults to requests package default.	None
over_client		client to overwrite configuration with	None

Returns:

Type	Description
ClientBaseWithDatastack	List of datastack names for available datastacks on the annotation engine