Replace the pyMBE Pandas dataframe for a canonical pyMBE database (#147)

* first draft

* Database with reactions and states disentangled

* first complete draft of the database for particles

* add all pmb_types

* clean up and document storage code

* start refactoring pyMBE to use the new database

* updated residue and bond management in pyMBE

* finish refactoring of residues, molecules and peptides

* updated create_protein

* refactor hydrogels

* update delete methods to work with the new database

* fix added_salt and counterion methods and get_id_map

* update methods to work with the new database

* fix test for define and create molecules

* modernize molecule tests, start cleaning lj tests

* implement lj templates, fix tests

* add demo, add functions to load/save the database to the pyMBE API

* fix unit test

* fix bug in create_counterions, fix unittest

* update copyright

* disentagle particle states from particle templates

* fix wrong handling of espresso types and acidity test

* fix molecule position test

* fix test

* fix seed test and load_interaction_parameters

* add metadata management, small cleanup

* clean up gcmc functional test

* fix bugs in setup bond, fix unittest

* fix net charge test

* fix parameter test

* fix HH functions and tests, improve the functionality of calculate_net_charge for hydrogels, fix related bugs

* start refactoring protein unit test

* fix protein framework and unit test

* adapt cpH method

* adapt reaction methods to the new database, fix docs

* standarize docs

* clean up repo, fix bug in setup of rigid object

* increase coverage

* improve coverage, clean docs

* add missing test

* increase coverage manager io, fix bug in lj

* remove dead ends in database io

* improve docs

* increase coverage of the database manager

* reach full coverage

* solve issues detected by pylint

* fix bug

* solve dependency issues on the CI

* fix new bug in unit test

* lower requierements for pydantic to comply with current EESSI

* fix hydrogel matplotlib representation, fix bug in collect_particle_templates with nested residues, continue refactoring samples

* fix general samples

* all functional tests run now, started fixing the tutorial

* clean up tutorial

* update changelog

* adress review comments by sebas and somesh

* fix affiliations

* update changelog

* address ipsita comment

* change behaviour of create_ methods to raise a ValueError if no template is found

* fix missing commit number in changelog

---------

Co-authored-by: pmblanco <pablomiguel.blanco@udl.cat>
Co-authored-by: Pablo <pablb.ntnu.no>

Author: pm-blanco

AUTHORS.md

@@ -5,12 +5,12 @@ The following people have contributed to pyMBE (Github username, Name, current a
 ## Admins
 - jngrad, Jean-Noël Grad (University of Stuttgart, Germany)
 - kosovan, Peter Košovan (Charles University, Czech Republic)
-- pm-blanco, Pablo M. Blanco (Norwegian University of Science and Technology, Norway)
+- pm-blanco, Pablo M. Blanco (University of Lleida, Spain)
 
 ## Tier-0 Contributors
 - davidbbeyer, David Beyer (University of Stuttgart, Germany)
 - paobtorres, Paola B. Torres (Universidad Tecnológica Nacional, Argentina)
-- pinedaps, Sebastian P. Pineda (Charles University, Czech Republic)
+- pinedaps, Sebastian P. Pineda (University of Lund, Sweden)
 - 1234somesh, Somesh Kurahatti (University of Stuttgart, Germany)
 
 ## Tier-1 Contributors

CHANGELOG.md

@@ -7,10 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-### Changed
-- Methods that interact directly with the pyMBE dataframe are now private and stored in a dedicated module in `storage/df_management`. These methods also have been refactored to be stateless methods, i.e. making it impossible for them to change behavior during the pyMBE object lifetime or for the user to change the pyMBE dataframe unless explicitely calling them. This includes the methods: `add_bond_in_df`, `add_value_to_df`, `assign_molecule_id`, `check_if_df_cell_has_a_value`, `check_if_name_is_defined_in_df`, `check_if_multiple_pmb_types_for_name`, `clean_df_row`, `clean_ids_in_df_row`, `copy_df_entry`, `create_variable_with_units`, `convert_columns_to_original_format`, `convert_str_to_bond_object`, `delete_entries_in_df`, `find_bond_key`, `setup_df`. (#145)
-- `define_particle_entry_in_df` is now a private method in pyMBE, as it is a convenience method for internal use. (#145)
-- The custom `NumpyEncoder` is now a private class in the private module `storage/df_management` because it is  only internally used in pyMBE for serialization/deserialization. (#145)
+## Added
+- Introduced a canonical pyMBE database backend replacing the previous monolithic Pandas DataFrame storage approach. This lays the foundation for more robust, extensible, and normalized data handling across pyMBE. (#147)
+- Added support to define reaction templates  in the pyMBE database. (#147)
+- Utility functions to cast information about templates and instances in the pyMBE database into pandas dataframe `pmb.get_templates_df`, `pmb.get_instances_df` and `pmb.get_reactions_df`. (#147)
+- Utility functions to load and save the new database via the pyMBE API, `pmb.save_database` and `pmb.load_database`. (#147)
+- Added functions to define particle states:  `pmb.define_particle_states`  and `pmb.define_monoprototic_particle_states`. (#147)
+- Added utility functions in `lib/handy_functions` to define residue and particle templates for aminoacids en peptides and residues: `define_protein_AA_particles`, `define_protein_AA_residues` and `define_peptide_AA_residues`. (#147)
+
+## Changed
+- Create methods (`create_particle`, `create_residue`, `create_molecule`, `create_protein`, `create_hydrogel`) now raise a ValueError if no template is found for an input `name` instead than a warning. (#147)
+- Refactored core modules to use the new database schema based on templates and instances  for particles, residues, molecules, hydrogels, proteins and peptides. (#147)
+- Particle states now are independent templates and are now disentangled from particle templates. (#147)
+- Pka values are now stored as part of chemical reactions and no longer an attribute of particle templates. (#147)
+- Amino acid residue templates are no longer defined internally in `define_peptide` and `define_protein`. Those definitions are now exposed to the user. (#147)
+- Molecule templates now need to be defined to be used as templates for hydrogel chains in hydrogels. (#147)
+
+## Fixed
+- Wrong handling of units in `get_radius_map` when the `dimensionless` argument was triggered. (#147)
+- Utility methods `get_particle_id_map`, `calculate_HH`, `calculate_net_charge`,  `center_object_in_simulation_box` now support all template types in pyMBE, including hydrogels. Some of these methods have been renamed to expose directly in the API this change in behavior. (#147)
+
+
+### Removed
+- Methods that interact directly with the pyMBE dataframe. These methods have been replaced by private methods that instead interact with the new canonical pyMBE database in (`pyMBE/storage/manager`). This includes the methods: `add_bond_in_df`, `add_value_to_df`, `assign_molecule_id`, `check_if_df_cell_has_a_value`, `check_if_name_is_defined_in_df`, `check_if_multiple_pmb_types_for_name`, `clean_df_row`, `clean_ids_in_df_row`, `copy_df_entry`, `create_variable_with_units`, `convert_columns_to_original_format`, `convert_str_to_bond_object`, `delete_entries_in_df`, `find_bond_key`, `setup_df`, `define_particle_entry_in_df`, custom `NumpyEncoder`. (#145,#147)
+- Method `add_bonds_to_espresso` has been removed from the API. pyMBE now adds bonds internally to ESPResSo when molecule instances are created into ESPResSo. (#147)
+- Tutorial `lattice_builder.ipynb` has been removed because its content is redundant with sample script `build_hydrogel.py`. (#147)
+
 
 ## [1.0.0] - 2025-10-08
 

CONTRIBUTING.md

@@ -11,7 +11,7 @@ New developers are welcome to contribute to extend the functionalities of the py
 To contribute to the pyMBE module, first one needs to be added as a member of this GitHub repository.
 
 If you want to contribute to the development of the pyMBE module,
-please contact Dr. Pablo M. Blanco (<pablb@ntnu.no>).
+please contact Dr. Pablo M. Blanco ([@pm-blanco](https://github.com/pm-blanco)).
 
 ## Authorship and contributorship policy
 For more information on our authorship and contributorship policy,

README.md

@@ -232,7 +232,7 @@ If you use pyMBE in your research, please cite our paper:
 
 When using a released version of pyMBE, we recommend citing the corresponding
 [Zenodo record](https://doi.org/10.5281/zenodo.12102634) in addition to the pyMBE paper,
-for example: "We set up our coarse-grained models using pyMBE v0.8.0
+for example: "We set up our coarse-grained models using pyMBE v1.0.0
 \cite{beyer2024pymbe,zenodo2024pymbe}".
 
 Please also make sure to properly cite the original authors if you use the resources provided in the `pyMBE/parameters/` folder.
@@ -247,7 +247,7 @@ contribute to pyMBE and find our authorship policy and code of conduct.
 
 ## License
 
-Copyright (C) 2023-2025 pyMBE-dev team
+Copyright (C) 2023-2026 pyMBE-dev team
 
 pyMBE is free software: you can redistribute it and/or modify it
 under the terms of the GNU General Public License as published by