Extracting edits to the level2 graph

Get a more detailed change log for a root ID

We provide a utility function for combining the output of a CAVEclient's chunkedgraph.get_tabular_change_log and chunkedgraph.get_operation_details.

Note

This probably belongs in CAVEclient

from caveclient import CAVEclient
from paleo import get_detailed_change_log

root_id = 864691135639556411

client = CAVEclient("minnie65_phase3_v1")

change_log = get_detailed_change_log(root_id, client, filtered=False)
change_log.iloc[:, :5]

	user_id	before_root_ids	after_root_ids	is_merge	in_neuron
operation_id
9028	161	[864691135338600033, 864691135349337074]	[864691134884740858]	True	False
14768	161	[864691135437585906, 864691136156871317]	[864691136272932030]	True	False
25672	161	[864691135190922026, 864691135566303299]	[864691135493963920]	True	False
26481	161	[864691134843526581, 864691135573183940]	[864691135415463098]	True	False
30000	161	[864691134975250238, 864691135587889860]	[864691135476114088]	True	False
...	...	...	...	...	...
532312	2378	[864691135475323712]	[864691137054392182]	False	True
532314	2378	[864691137054392182]	[864691136089721911]	False	True
532327	2378	[864691136089721911]	[864691135975162223]	False	True
532335	2378	[864691135975162223]	[864691135851270855]	False	True
532354	2378	[864691135614747083, 864691135851270855]	[864691135639556411]	True	True

693 rows × 5 columns

Get the changes to the level2 graph for a specific edit

edit_id = change_log.index[0]
edit_id

9028

import time
from paleo import get_operation_level2_edit

currtime = time.time()
l2_edit = get_operation_level2_edit(edit_id, client)
print(f"{time.time() - currtime:.3f} seconds elapsed.")

4.092 seconds elapsed.

This returns a paleo.NetworkDelta object. This is a lightweight class for storing what edges/nodes are added/removed by an operation, as well as (optionally) a dictionary of arbitrary metadata.

l2_edit

NetworkDelta(
   removed_nodes: 26,
   added_nodes: 9,
   removed_edges: 63,
   added_edges: 33,
   metadata: {}
)

Get the changes to the level2 graph for multiple edits

With the current set of endpoints and design of the chunkedgraph, it is faster to get a list of changes using this function rather than putting paleo.get_operation_level2_edit in a loop.

from paleo import get_operations_level2_edits

currtime = time.time()
l2_edits = get_operations_level2_edits(change_log.index, client)
print(f"{time.time() - currtime:.3f} seconds elapsed.")

Extracting level2 edits:   0%|          | 0/693 [00:00<?, ?it/s]

122.348 seconds elapsed.

This returns a dictionary, where the keys are the operation IDs and the values are the corresponding paleo.NetworkDelta objects.

Get the changes to the level2 graph for all edits to a root ID

We get even more of a speedup if all of those changes are from the history of one root ID. Again, this is just because of the way current endpoints are structured (some of the information we need is available from CAVEclient.chunkedgraph.get_tabular_change_log).

from paleo import get_root_level2_edits

currtime = time.time()
l2_edits = get_root_level2_edits(root_id, client)
print(f"{time.time() - currtime:.3f} seconds elapsed.")

Extracting level2 edits:   0%|          | 0/693 [00:00<?, ?it/s]

59.575 seconds elapsed.

Combine edits that affect the same points on the level2 graph

Often it is helpful to combine edits that affect the same points on the level2 graph. This can be useful for finding things like edits that were undone by later edits.

Currently, this function defines metaedits as those that affect the same points on the level2 graph (more specifically, connected components in a graph where nodes are edits and edges are shared points on the level2 graph). In the future, this could support other schemes for defining metaedits.

from paleo import get_metaedits

metaedits, metaedit_mapping = get_metaedits(l2_edits)

metaedits[23]

NetworkDelta(
   removed_nodes: 5,
   added_nodes: 7,
   removed_edges: 17,
   added_edges: 19,
   metadata: {}
)

member_edits = metaedit_mapping[23]
for edit in member_edits:
    print(list(l2_edits[edit].added_nodes))

[161733831655687152]
[161663462911509525, 161663462911509526, 161733831655687154, 161733831655687155]
[161663462911509527, 161733831655687156]

metaedits[23].added_nodes

array([161733831655687152, 161663462911509525, 161663462911509526,
       161733831655687154, 161733831655687155, 161663462911509527,
       161733831655687156])