Graduation / changing protocols

[cxrodgers]

Is there an easy way to change the protocol.json that a Subject is using, without defining graduation criteria? The mouse hasn't actually reached any criterion, I just want to transition the same Subject to a new protocol that I created. In the past I have always just done this by creating a new Subject, but in this case I'd like to keep everything in the same HDF5 file if possible.

Also, on a related note, I see that "reward" is typically a parameter in the protocol.json. Is there a way to modify the reward value (or really any parameter) separately for each mouse? Again, the only way I've done this before is to create a separate protocol for each mouse, but this can lead to bugs because some parameters are supposed to be the same across mice and others are supposed to vary across mice.

thanks!

[sneakers-the-rat]

Yes!

• The Subject class should automatically detect changes to the .json protocol it is assigned:

  autopilot/autopilot/data/subject.py

  Lines 538 to 548 in 04b5968

  	def _check_protocol_changed(self):
  	"""Check whether the protocol on disk has changed. If it has, update!"""
  	try:
  	prot_name, disk_protocol = self._find_protocol(self.protocol_name)
  	except Exception as e:
  	self.logger.warning(f"Could not find protocol file to update internal representation of it. Got exception {e}")
  	return
  	if disk_protocol != self.protocol.protocol:
  	self.logger.info('Protocol on disk changed from stored protocol. Updating')
  	self.assign_protocol(disk_protocol, step_n=self.protocol.step, pilot=self.protocol.pilot, protocol_name=prot_name)

• The Terminal GUI has an action to batch reassign protocols (Tools > Batch Reassign Protocols) (

  autopilot/autopilot/gui/menus/tools.py

  Line 83 in 04b5968

  class Reassign(QtWidgets.QDialog):

  and

  autopilot/autopilot/agents/terminal.py

  Line 797 in 04b5968

  def reassign_protocols(self):

  )
• Individual Subjects can have their protocols reassigned directly from their protocol attribute ( https://docs.auto-pi-lot.com/en/latest/data/subject.html#autopilot.data.subject.Subject.protocol ) using a Protocol_Status object ( https://docs.auto-pi-lot.com/en/latest/data/models/subject.html#autopilot.data.models.subject.Protocol_Status ) or the assign_protocol method: https://docs.auto-pi-lot.com/en/latest/data/subject.html#autopilot.data.subject.Subject.assign_protocol

Parameters that vary across mice are a great idea, but not currently formally supported. This is something I had intended to write alongside the currently hardcoded pre/post-run dialogues for inputting weight. I think there are at least three types of task parameters that should be supported:

• Task-global parameters that are true for all subjects running a task, to be declared at protocol creation
• Subject-level parameters that vary for each subject running a task, to be declared at protocol assignment
• Session-level parameters that vary for each session, to be declared at start/stop of running a task.

I issued v0.5.0 as an alpha release because there are still a few places that need to be formalized in the new data modeling system, and task parameters are the major one. I'd see this being implemented like this:

• There's a stub for a Task_Params class ( https://docs.auto-pi-lot.com/en/latest/data/models/protocol.html#autopilot.data.models.protocol.Task_Params ) that would replace the PARAMS dictionary in a Task definition that should either
  • Support Field annotations to declare different parameter types, (which i favor for maintainability and less boilerplate) eg:

  class PARAMS(Task_Params):
      global_param: int = Field(description="Some task global param, the default type")
      subject_param: int = Field(
          description="Subject-specific param",
          param_scope="subject")
      session_param:int = Field(
          description="Param given every session",
          param_scope="session")

  • Require sub-models that carry each of the different scopes, eg:

  class Global_Params(Params):
      global_param: int
  
  class Subject_Params(Params):
      subject_param: int
  
  class Session_Params(Params):
      session_param: int
  
  class PARAMS(Task_Params):
      global = Global_Params
      subject = Subject_Params
      session = Session_Params

• Store these separate classes of parameters in separate attribute dictionaries in the relevant protocol node (see the .protocol setter method)
• Be exposed to editing from the GUI (using the model GUI widget: https://docs.auto-pi-lot.com/en/latest/gui/widgets/model.html ) and the subject class ( no extra code, just reassigning the protocol property)

Would that cover what you have in mind?

[sneakers-the-rat]

There's also a very in-progress effort to make the subject data model general and modifiable from plugins (

autopilot/autopilot/data/models/subject.py

Lines 121 to 162 in 04b5968

	class Subject_Structure(Schema):
	"""
	Structure of the :class:`.Subject` class's hdf5 file
	"""
	info = H5F_Group(path='/info', title="Subject Biographical Information")
	data = H5F_Group(path='/data', filters=tables.Filters(complevel=6, complib='blosc:lz4'))
	protocol = H5F_Group(path='/protocol', title="Metadata for the currently assigned protocol")
	history = H5F_Group(path='/history', children=[
	H5F_Group(path='/history/past_protocols', title='Past Protocol Files'),
	_Hash_Table(path='/history/hashes', title="Git commit hash history"),
	_History_Table(path='/history/history', title="Change History"),
	_Weight_Table(path='/history/weights', title="Subject Weights")
	])
	def make(self, h5f:tables.file.File):
	"""
	Make all the nodes!
	Args:
	h5f (:class:`tables.file.File`): The h5f file to make the groups in!
	"""
	for _, node in self._iter():
	node.make(h5f)
	class Subject_Schema(Schema):
	"""
	Structure of the :class:`.Subject` class's hdf5 file
	.. todo::
	Convert this into an abstract representation of data rather than literally hdf5 tables.
	At the moment twins :class:`.Subject_Structure`
	"""
	info: Biography
	data: Protocol_Data
	protocol: Protocol_Status
	past_protocols: List[Protocol_Status]
	history: History_Group

), and again an unpopulated stub module for making links between data model fields (autopilot.data.modeling.links) so that it would be possible to do this without explicit support, so you would be able to say the reward parameter is linked to some subject-specific attribute node in its hdf5 file like reward: units.mL = Link('root.task_info.reward').

I had also intended to create a Reward_Manager class (like the very badly implemented Stim_Manager) that could let rewards be managed as the result of a subject's history in the task: eg. to specify that a subject should get a total amount of reward per session, etc. and that would also be much much easier to write now with a formal data modeling system.

would love to help write/advise/etc. whatever would be the most useful thing to you

[cxrodgers]

"Batch Reassign Protocols" is great!! I can't believe I never noticed that before. Everyone here is excited about how this will things easier for us.

I like the idea of the global/subject/session params. I'll think more about how this would work.