{ "cells": [ { "cell_type": "markdown", "id": "73926310", "metadata": {}, "source": [ "## DDBST fragmentation database" ] }, { "cell_type": "markdown", "id": "1748c46f", "metadata": {}, "source": [ "In this notebook, we evaluate the performance of `ugropy` against the DDBST\n", "fragmentation database. The database file was originally published in Simon\n", "Müller's article\n", "([https://doi.org/10.1186/s13321-019-0382-3](https://doi.org/10.1186/s13321-019-0382-3))\n", "and is hosted in the official repository:\n", "[https://github.com/simonmb/fragmentation_algorithm](https://github.com/simonmb/fragmentation_algorithm).\n", "\n", "To run these tests, the database file has been replicated in the `ugropy`\n", "repository. All rights to the original data are retained by the authors. Both\n", "this notebook and the database file are available at:\n", "[https://github.com/ipqa-research/ugropy/tree/main/docs/source/tests_book/against_ddbst](https://github.com/ipqa-research/ugropy/tree/main/docs/source/tests_book/against_ddbst)" ] }, { "cell_type": "markdown", "id": "f23c3916", "metadata": {}, "source": [ "The database file in the `ugropy` repository has been slightly modified by\n", "adding custom comments for tracking purposes; however, this notebook can be\n", "executed using the original, unmodified database file with identical results.\n", "\n", "These comments were added to the \"code\" column of the file to identify specific\n", "edge cases or behaviors for certain molecules of interest. The documentation\n", "for these characters can be found at the beginning of the database file, and\n", "their meanings are defined as follows:\n", "\n", "* `?`: **Disagreement with the database solution.** Indicates cases where\n", " `ugropy` yields a different fragmentation than the DDBST solution, and the\n", " database solution appears questionable. For the purpose of this test, these\n", " molecules are still accounted for as failed solutions.\n", "\n", "* `??`: **Combinatorial explosion / Slow multi-solution retrieval.** `ugropy`\n", " fails to obtain the correct solution on its first attempt. Reaching the\n", " correct solution requires exploring all possible alternative fragmentation\n", " pathways. Since these molecules exhibit a high number of symmetric\n", " multi-solutions, the computational time required to resolve them all is\n", " significantly high. Consequently, these molecules are skipped in this test\n", " and counted as failed solutions, even though the correct solution could\n", " eventually be reached with sufficient time.\n", "\n", "* `?!`: **Aromaticity mismatches.** The RDKit library detects aromatic rings\n", " that DDBST considers non-aromatic, which inevitably leads to a mismatch in\n", " the group assignment. In this test, these molecules are considered failed\n", " solutions." ] }, { "cell_type": "code", "execution_count": 1, "id": "58ec6fd6", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
codenumbersolution
smiles
CC1=C(C(=CC(=C1Cl)Cl)Cl)ClZOXPZFFPPKVNEA-UHFFFAOYSA-N138709:1|11:1|53:4
CCC=CCC1=C(CCC1=O)CXMLSXPIVAXONDL-UHFFFAOYSA-N102611:2|2:3|19:1|70:1|6:1
C#CCClLJZPPWWHKPGCHS-UHFFFAOYSA-N1222165:1|44:1
C1=CN=CC=C1CCNIDLHTECVNDEOIY-UHFFFAOYSA-N832752:1|29:1|38:1
C[Si](C)(C)C[Si](C)(C)CClWBZTYASEAQTKBK-UHFFFAOYSA-N875741:5|2:1|44:1|81:2
............
CN(C)C1=CC=CC2=C1C=CC=C2N(C)CTYMBOHQMUQAOBA-UHFFFAOYSA-N1390881:2|10:4|34:2|9:6
C=CCCCCCCCCCC=CBPHFKBMQSYYNGQ-UHFFFAOYSA-N308742:9|5:2
C=CC#N.C1=CC(=CC(=C1)CN)CNXMEXUJUMKRUUNG-UHFFFAOYSA-N713113419:4|10:2|68:1|29:2
CCN(CC)CCOC(=O)C1=C(C=C(C=C1)N)OGHSCYMOJHVOGDJ-UHFFFAOYSA-N680971:2|2:3|35:1|36:1|9:3|10:1|77:1|17:1
CC1CC(C2=CC=CC=C12)CIIJUYSSJMAITHJ-UHFFFAOYSA-N201431:2|2:1|13:2|9:4
\n", "

28675 rows × 3 columns

\n", "
" ], "text/plain": [ " code number \\\n", "smiles \n", "CC1=C(C(=CC(=C1Cl)Cl)Cl)Cl ZOXPZFFPPKVNEA-UHFFFAOYSA-N 13870 \n", "CCC=CCC1=C(CCC1=O)C XMLSXPIVAXONDL-UHFFFAOYSA-N 10261 \n", "C#CCCl LJZPPWWHKPGCHS-UHFFFAOYSA-N 12221 \n", "C1=CN=CC=C1CCN IDLHTECVNDEOIY-UHFFFAOYSA-N 83275 \n", "C[Si](C)(C)C[Si](C)(C)CCl WBZTYASEAQTKBK-UHFFFAOYSA-N 87574 \n", "... ... ... \n", "CN(C)C1=CC=CC2=C1C=CC=C2N(C)C TYMBOHQMUQAOBA-UHFFFAOYSA-N 139088 \n", "C=CCCCCCCCCCC=C BPHFKBMQSYYNGQ-UHFFFAOYSA-N 30874 \n", "C=CC#N.C1=CC(=CC(=C1)CN)CN XMEXUJUMKRUUNG-UHFFFAOYSA-N 71311341 \n", "CCN(CC)CCOC(=O)C1=C(C=C(C=C1)N)O GHSCYMOJHVOGDJ-UHFFFAOYSA-N 68097 \n", "CC1CC(C2=CC=CC=C12)C IIJUYSSJMAITHJ-UHFFFAOYSA-N 20143 \n", "\n", " solution \n", "smiles \n", "CC1=C(C(=CC(=C1Cl)Cl)Cl)Cl 9:1|11:1|53:4 \n", "CCC=CCC1=C(CCC1=O)C 1:2|2:3|19:1|70:1|6:1 \n", "C#CCCl 65:1|44:1 \n", "C1=CN=CC=C1CCN 2:1|29:1|38:1 \n", "C[Si](C)(C)C[Si](C)(C)CCl 1:5|2:1|44:1|81:2 \n", "... ... \n", "CN(C)C1=CC=CC2=C1C=CC=C2N(C)C 1:2|10:4|34:2|9:6 \n", "C=CCCCCCCCCCC=C 2:9|5:2 \n", "C=CC#N.C1=CC(=CC(=C1)CN)CN 9:4|10:2|68:1|29:2 \n", "CCN(CC)CCOC(=O)C1=C(C=C(C=C1)N)O 1:2|2:3|35:1|36:1|9:3|10:1|77:1|17:1 \n", "CC1CC(C2=CC=CC=C12)C 1:2|2:1|13:2|9:4 \n", "\n", "[28675 rows x 3 columns]" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from ugropy import unifac\n", "\n", "import pandas as pd\n", "\n", "# Skip the comment lines at the beginning of the file\n", "df = pd.read_csv(\"ddbst_tests.csv\", index_col=1, skiprows=3)\n", "\n", "df" ] }, { "cell_type": "markdown", "id": "e3604432", "metadata": {}, "source": [ "### Auxiliary function" ] }, { "cell_type": "markdown", "id": "0051c81f", "metadata": {}, "source": [ "The following function converts the DDBST fragmentation strings into a\n", "dictionary format compatible with `ugropy` outputs. While the DDBST database\n", "provides fragmentation results as a serialized string of group IDs and counts,\n", "`ugropy` returns a Python dictionary. This function parses the DDBST string\n", "into a matching dictionary to facilitate a direct comparison.\n", "\n", "**Example:**\n", "\n", "For a given DDBST string representing a molecule's functional groups:\n", "`\"9:1|11:1|53:4\"`\n", "\n", "The function processes it as follows:\n", "\n", "```\n", "\"9:1|11:1|53:4\" -> get_sol_dict -> {\"ACH\": 1, \"ACCH3\": 1, \"ACCL\": 4}\n", "```" ] }, { "cell_type": "code", "execution_count": 2, "id": "a48662fa", "metadata": {}, "outputs": [], "source": [ "# Function to convert the solution string into a dictionary of group \n", "# occurrences. E.g. 9:1|11:1|53:4 -> {'ACH': 1, 'ACCH3': 1, 'ACCL': 4}\n", "subgroup_lookup = (\n", " unifac.subgroups_info\n", " .reset_index()\n", " .set_index(\"subgroup_number\")[\"group\"]\n", " .to_dict()\n", ")\n", "\n", "def get_sol_dict(solution: str):\n", " return {\n", " subgroup_lookup[int(g)]: int(o)\n", " for g, o in (group.split(\":\") for group in solution.split(\"|\"))\n", " }" ] }, { "cell_type": "markdown", "id": "ced0d6cc", "metadata": {}, "source": [ "### First-attempt success rate" ] }, { "cell_type": "markdown", "id": "5830b376", "metadata": {}, "source": [ "Due to the nature of the `ugropy` fragmentation algorithm, certain molecules\n", "may exhibit solution multiplicity. In these cases, the first generated solution\n", "might not match the expected one, even though the correct fragmentation can be\n", "obtained by exploring all the solutions. In this specific test,\n", "we evaluate only the success rate of the **first obtained solution**.\n", "\n", "The following counters and lists track the benchmarking results:\n", "\n", "* `count_total`: Total number of molecules evaluated (at the end is the same as\n", " the total rows processed from the dataframe).\n", "\n", "* `count_first_correct`: Number of molecules where the first solution obtained\n", " by `ugropy` matches the DDBST database.\n", "\n", "* `count_skipped`: Number of molecules skipped due to the `??` tag (tracked as\n", " failed solutions for this test).\n", "\n", "* `smiles_incorrects`: SMILES strings of the molecules that failed on the first\n", " attempt. These are stored for later analysis to determine if they can be\n", " resolved via multi-solution search.\n", "\n", "* `sol_expected_incorrects`: The expected DDBST solution dictionaries for the\n", " failed cases, stored to avoid parsing the raw strings again during\n", " post-analysis." ] }, { "cell_type": "code", "execution_count": 3, "id": "9cd8dcbe", "metadata": {}, "outputs": [], "source": [ "count_total = 0 # Total number of solutions compared\n", "count_first_correct = 0 # Number of times the first solution matches\n", "count_skipped = 0 # Number of solutions skipped due to \"??\"\n", "smiles_incorrects = [] # List of SMILES for which the first solution is incorrect\n", "sol_expected_incorrects = [] # List of expected solutions for the incorrect cases\n", "\n", "for smiles, code, solution in zip(df.index, df[\"code\"], df[\"solution\"]):\n", "\n", " count_total += 1\n", "\n", " if \"??\" in code:\n", " count_skipped += 1\n", " continue\n", "\n", " ddbst_sol = get_sol_dict(solution)\n", " ugropy_sol = unifac.get_groups(smiles, \"smiles\").subgroups\n", "\n", " if ddbst_sol == ugropy_sol:\n", " count_first_correct += 1\n", " else:\n", " smiles_incorrects.append(smiles)\n", " sol_expected_incorrects.append(ddbst_sol)" ] }, { "cell_type": "markdown", "id": "e910e71f", "metadata": {}, "source": [ "Following the initial evaluation (which takes approximately 2 minutes on a\n", "standard desktop computer), we can compute the primary performance metrics for\n", "`ugropy`:\n", "\n", "* `count_compared`: The number of molecules evaluated in the comparison (total\n", " molecules minus those skipped).\n", "\n", "* `count_incorrect`: The number of molecules that failed on the first attempt\n", " (the sum of the `smiles_incorrects` list plus the skipped molecules).\n", "\n", "* `pct_correct`: The first-attempt success rate, calculated as:\n", " `count_first_correct / count_total * 100`.\n", "\n", "* `pct_incorrect`: The first-attempt failure rate, calculated as:\n", " `count_incorrect / count_total * 100`." ] }, { "cell_type": "code", "execution_count": 4, "id": "fb7a83c2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "=============================================\n", " UGROPY VALIDATION RESULTS \n", "=============================================\n", " Total molecules: 28675\n", " Skipped (??): 29\n", " Compared: 28646\n", "---------------------------------------------\n", " ✅ Correct: 26567 (92.649%)\n", " ❌ Incorrect: 2108 (7.351%)\n", "=============================================\n" ] } ], "source": [ "count_compared = count_total - count_skipped\n", "count_incorrect = len(smiles_incorrects) + count_skipped\n", "pct_correct = count_first_correct / count_total * 100\n", "pct_incorrect = count_incorrect / count_total * 100\n", "\n", "print(\"=\" * 45)\n", "print(f\"{'UGROPY VALIDATION RESULTS':^45}\")\n", "print(\"=\" * 45)\n", "print(f\" Total molecules: {count_total:>6}\")\n", "print(f\" Skipped (??): {count_skipped:>6}\")\n", "print(f\" Compared: {count_compared:>6}\")\n", "print(\"-\" * 45)\n", "print(f\" ✅ Correct: {count_first_correct:>6} ({pct_correct:.3f}%)\")\n", "print(f\" ❌ Incorrect: {count_incorrect:>6} ({pct_incorrect:.3f}%)\")\n", "print(\"=\" * 45)\n" ] }, { "cell_type": "markdown", "id": "33e1fd23", "metadata": {}, "source": [ "### Solution multiplicity success rate" ] }, { "cell_type": "markdown", "id": "8ceb4189", "metadata": {}, "source": [ "From the solutions that failed on the first attempt, we can analyze how many of\n", "them can be resolved by exploring all the alternative solutions generated by\n", "`ugropy`. This analysis provides insight into the algorithm's ability to\n", "eventually find the correct fragmentation, even if it is not the first choice.\n", "In this analysis, a solution is considered correct if the expected DDBST\n", "solution is found within any of the alternative paths generated by `ugropy`.\n", "This process takes around 1 minute on a standard desktop computer.\n", "\n", "The following counters and lists track the results of this analysis:\n", "\n", "* `count_multiplicity_correct`: The number of molecules that were initially\n", " incorrect but could be resolved by exploring all alternative solutions\n", " (additional correct solutions found).\n", "\n", "* `store_multiple_solutions`: A list that stores the alternative solutions\n", " generated by `ugropy` where one of them was correct. This is used later to\n", " test the different multi-solution filters that `ugropy` provides (avoiding\n", " the need to search for them again in subsequent tests).\n", "\n", "* `smiles_multiple_correct`: The SMILES strings of the molecules successfully\n", " resolved by exploring all solutions. This is also used later to test the\n", " different multi-solution filters.\n", "\n", "* `expected_multiple_correct`: The expected DDBST solution dictionaries for the\n", " molecules successfully resolved by exploring all solutions. This is also\n", " stored to test the different multi-solution filters later.\n", "\n", "* `incorrects_smiles`: The SMILES strings of the molecules that remain\n", " incorrect even after exploring all alternative solutions.\n", "\n", "* `incorrects_expected`: The expected DDBST solution dictionaries for the\n", " molecules that remain incorrect even after exploring all alternative\n", " solutions." ] }, { "cell_type": "code", "execution_count": 5, "id": "44a9a1c1", "metadata": {}, "outputs": [], "source": [ "count_multiplicity_correct = 0 # Counts correct solutions for molecules with solutions multiplicity\n", "store_multiple_solutions = [] # List multiple solutions for filters testing\n", "smiles_multiple_correct = [] # Store SMILES of molecules with multiple correct solutions\n", "expected_multiple_correct = [] # Store expected solutions of molecules with multiple correct solutions\n", "incorrects_smiles = [] # Store SMILES of definitely incorrect solutions\n", "incorrects_expected = [] # Store expected solutions of definitely incorrect solutions\n", "\n", "for smiles, expected in zip(smiles_incorrects, sol_expected_incorrects):\n", " solutions = unifac.get_groups(smiles, \"smiles\", search_multiple_solutions=True)\n", " sols_groups = [sol.subgroups for sol in solutions]\n", " \n", " if expected in sols_groups:\n", " count_multiplicity_correct += 1\n", " smiles_multiple_correct.append(smiles)\n", " expected_multiple_correct.append(expected)\n", " store_multiple_solutions.append(solutions)\n", " else:\n", " incorrects_smiles.append(smiles)\n", " incorrects_expected.append(expected)" ] }, { "cell_type": "markdown", "id": "ea72b647", "metadata": {}, "source": [ "After this secondary evaluation, we can analyze the following cumulative metrics:\n", "\n", "* `count_total_correct`: The total number of successfully fragmented molecules,\n", " either on the first attempt or through multi-solution search, calculated as:\n", " `count_first_correct + count_multiplicity_correct`.\n", "\n", "* `count_definitely_incorrect`: The number of molecules that remain unresolved\n", " even after exploring all alternative paths, calculated as the length of\n", " `incorrects_smiles`. Note that skipped molecules are excluded here since they\n", " were not processed in the multi-solution search loop.\n", "\n", "* `count_total_incorrect`: The total number of failed cases, combining both\n", " unresolved molecules and skipped ones, calculated as:\n", " `count_definitely_incorrect + count_skipped`.\n", "\n", "* `pct_total_correct`: The overall success rate considering both first-attempt\n", " matches and multi-solution resolutions, calculated as: \n", " `count_total_correct / count_total * 100`.\n", "\n", "* `pct_multi`: The percentage of total dataset molecules that were successfully\n", " resolved specifically via multi-solution search, calculated as:\n", " `count_multiplicity_correct / count_total * 100`. This metric reflects the\n", " fraction of the complete dataset that required exploring alternative pathways\n", " to achieve a correct fragmentation." ] }, { "cell_type": "code", "execution_count": 6, "id": "bb97b468", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "=============================================\n", " UGROPY VALIDATION RESULTS \n", "=============================================\n", " Total molecules: 28675\n", " Skipped (??): 29\n", " Compared: 28646\n", "---------------------------------------------\n", " ✅ Correct (first): 26567 (92.649%)\n", " ✅ Correct (multiple): 1817 (6.337%)\n", " ───────────────────\n", " ✅ Total correct: 28384 (98.985%)\n", " ❌ Incorrect: 291 (1.015%)\n", "=============================================\n" ] } ], "source": [ "count_total_correct = count_first_correct + count_multiplicity_correct\n", "count_definitely_incorrect = len(incorrects_smiles)\n", "count_total_incorrect = count_definitely_incorrect + count_skipped\n", "pct_total_correct = count_total_correct / count_total * 100\n", "pct_multi = count_multiplicity_correct / count_total * 100\n", "pct_not_correct = count_total_incorrect / count_total * 100\n", "\n", "print(\"=\" * 45)\n", "print(f\"{'UGROPY VALIDATION RESULTS':^45}\")\n", "print(\"=\" * 45)\n", "print(f\" Total molecules: {count_total:>6}\")\n", "print(f\" Skipped (??): {count_skipped:>6}\")\n", "print(f\" Compared: {count_compared:>6}\")\n", "print(\"-\" * 45)\n", "print(f\" ✅ Correct (first): {count_first_correct:>9} ({pct_correct:.3f}%)\")\n", "print(f\" ✅ Correct (multiple): {count_multiplicity_correct:>6} ({pct_multi:.3f}%)\")\n", "print(f\" {'─'*19}\")\n", "print(f\" ✅ Total correct: {count_total_correct:>9} ({pct_total_correct:.3f}%)\")\n", "print(f\" ❌ Incorrect: {count_total_incorrect:>9} ({pct_not_correct:.3f}%)\")\n", "print(\"=\" * 45)" ] }, { "cell_type": "markdown", "id": "022ae3c4", "metadata": {}, "source": [ "### Testing ugropy multi-solution filters" ] }, { "cell_type": "markdown", "id": "edd5e802", "metadata": {}, "source": [ "Now we are going to evaluate the different multi-solution filters provided by\n", "`ugropy`. The objective is to determine which filter performs best at selecting\n", "the correct solution from the multiple alternatives generated by the algorithm.\n", "For a detailed explanation of how each filter operates, please refer to the\n", "tutorial documentation.\n", "\n", "The variables `smiles_multiple_correct` and `expected_multiple_correct` contain\n", "the molecules that required a multi-solution search to reach the correct\n", "fragmentation (SMILES and expected solution). Additionally,\n", "`store_multiple_solutions` holds the alternative solutions generated by\n", "`ugropy` for these specific molecules.\n", "\n", "The next variable `count_multiple_needed` represents the total number of\n", "molecules that required a multi-solution search. For each filter, we will count\n", "how many of these cases are correctly resolved and compute their respective\n", "success rates.\n", "\n", "In some cases, a filter may fail to select a unique correct solution.\n", "Naturally, those ambiguous cases are not counted as successful resolutions, but\n", "the corresponding code block can be uncommented to print them for further\n", "analysis. The other way of obtaining a fail case is when filters retain an\n", "unique solution but, of course, is the incorrect one." ] }, { "cell_type": "code", "execution_count": 7, "id": "54f6bdff", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of molecules that needed multiplicity to be correct: 1817\n" ] } ], "source": [ "count_multiple_needed = len(smiles_multiple_correct)\n", "\n", "print(f\"Number of molecules that needed multiplicity to be correct: {count_multiple_needed}\")" ] }, { "cell_type": "markdown", "id": "96bb9aff", "metadata": {}, "source": [ "First we will evaluate the performance of the base `FragmentationModel` filters\n", "which are not specifically designed for a `GibbsModel` (spoiler: they are very\n", "bad for UNIFAC)." ] }, { "cell_type": "markdown", "id": "09edbaa7", "metadata": {}, "source": [ "#### filter_mostly_polyatomic" ] }, { "cell_type": "code", "execution_count": 8, "id": "535c60f4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_mostly_polyatomic: 0/1817 (0.00%)\n" ] } ], "source": [ "success_mostly_polyatomic = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_mostly_polyatomic(sols)\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions: {[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_mostly_polyatomic += 1\n", " \n", "print(\n", " \"Success rate of filter_mostly_polyatomic: \"\n", " f\"{success_mostly_polyatomic}/{count_multiple_needed}\"\n", " f\" ({success_mostly_polyatomic/count_multiple_needed*100:.2f}%)\"\n", ")" ] }, { "cell_type": "markdown", "id": "ccfbf46b", "metadata": {}, "source": [ "#### filter_mostly_polarity — polar" ] }, { "cell_type": "code", "execution_count": 9, "id": "3c53a5a0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_mostly_polarity (polar): 78/1817 (4.29%)\n" ] } ], "source": [ "success_mostly_polarity__polar = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_mostly_polarity(sols, polarity=\"polar\")\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions: {[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_mostly_polarity__polar += 1\n", " \n", "print(\n", " \"Success rate of filter_mostly_polarity (polar): \"\n", " f\"{success_mostly_polarity__polar}/{count_multiple_needed}\"\n", " f\" ({success_mostly_polarity__polar/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "4729019b", "metadata": {}, "source": [ "#### filter_mostly_polarity — apolar" ] }, { "cell_type": "code", "execution_count": 10, "id": "9f4347a0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_mostly_polarity (apolar): 0/1817 (0.00%)\n" ] } ], "source": [ "success_mostly_polarity__apolar = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_mostly_polarity(sols, polarity=\"apolar\")\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions: {[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_mostly_polarity__apolar += 1\n", " \n", "print(\n", " \"Success rate of filter_mostly_polarity (apolar): \"\n", " f\"{success_mostly_polarity__apolar}/{count_multiple_needed}\"\n", " f\" ({success_mostly_polarity__apolar/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "02c2449c", "metadata": {}, "source": [ "Now we are going to evaluate the performance of the multi-solution filter\n", "specifically designed for Gibbs models." ] }, { "cell_type": "markdown", "id": "3fff5782", "metadata": {}, "source": [ "#### filter_bigger_polyatomics — Q" ] }, { "cell_type": "markdown", "id": "9a7c4fcf", "metadata": {}, "source": [ "The $Q$ parameters (surface area fractions) of each subgroup are highly\n", "relevant because they weight the residual term of the excess Gibbs free energy\n", "models." ] }, { "cell_type": "code", "execution_count": 11, "id": "9c325799", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_bigger_polyatomics: 1732/1817 (95.32%)\n" ] } ], "source": [ "success_biggers_polyatomics__Q = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_bigger_polyatomics(sols, criteria=\"Q\")\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions: {[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_biggers_polyatomics__Q += 1\n", " \n", "print(\n", " \"Success rate of filter_bigger_polyatomics: \"\n", " f\"{success_biggers_polyatomics__Q}/{count_multiple_needed}\"\n", " f\" ({success_biggers_polyatomics__Q/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "d88dcd4e", "metadata": {}, "source": [ "#### filter_bigger_polyatomics — R" ] }, { "cell_type": "code", "execution_count": 12, "id": "01470603", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_bigger_polyatomics: 1631/1817 (89.76%)\n" ] } ], "source": [ "success_biggers_polyatomics__R = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_bigger_polyatomics(sols, criteria=\"R\")\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions: {[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_biggers_polyatomics__R += 1\n", " \n", "print(\n", " \"Success rate of filter_bigger_polyatomics: \"\n", " f\"{success_biggers_polyatomics__R}/{count_multiple_needed}\"\n", " f\" ({success_biggers_polyatomics__R/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "b3dd637b", "metadata": {}, "source": [ "#### filter_polarity_contribution — Q — polar" ] }, { "cell_type": "markdown", "id": "21075ce5", "metadata": {}, "source": [ "Definitely the most accurate heuristic for UNIFAC implemented in `ugropy` at the\n", "moment." ] }, { "cell_type": "code", "execution_count": 13, "id": "96c048b1", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "SMILES: CCC(C)OCOC(C)CC\n", "Expected solution: {'CH3': 4, 'CH2': 3, 'CHO': 2}\n", "Filtered solution: {'CH3': 4, 'CH2': 2, 'CH': 1, 'CH2O': 1, 'CHO': 1}\n", "All solutions:[{'CH3': 4, 'CH2': 2, 'CH': 1, 'CH2O': 1, 'CHO': 1}, {'CH3': 4, 'CH2': 3, 'CHO': 2}]\n", "----------------------------------------------------------------------------------------------------\n", "SMILES: CN(CC1COC2=CC=CC=C2O1)C(=O)N(CC=C)CC=C\n", "Expected solution: {'CH2': 1, 'CH2N': 1, 'CH2=CH': 2, 'ACH': 4, 'AC': 2, 'AMCH3CH2': 1, 'CHO': 1, 'THF': 1}\n", "All solutions:[{'CH3': 1, 'CH2=CH': 2, 'ACH': 4, 'AC': 2, 'CHO': 1, 'THF': 1, 'CH2N': 1, 'AM(CH2)2': 1}, {'CH2': 1, 'CH2=CH': 2, 'ACH': 4, 'AC': 2, 'CHO': 1, 'THF': 1, 'CH2N': 1, 'AMCH3CH2': 1}, {'CH2': 1, 'CH2=CH': 2, 'ACH': 4, 'AC': 2, 'CHO': 1, 'THF': 1, 'CH3N': 1, 'AM(CH2)2': 1}]\n", "----------------------------------------------------------------------------------------------------\n", "Success rate of filter_polarity_contribution: 1815/1817 (99.89%)\n" ] } ], "source": [ "success_polarity_contribution__Q__polar = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_polarity_contribution(sols, criteria=\"Q\", polarity=\"polar\")\n", " \n", " if len(filtered) > 1:\n", " print(f\"SMILES: {smiles}\")\n", " print(f\"Expected solution: {expected}\")\n", " print(f\"All solutions:{[sol.subgroups for sol in sols]}\")\n", " print(f\"-\"*100)\n", " continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_polarity_contribution__Q__polar += 1\n", " else:\n", " print(f\"SMILES: {smiles}\")\n", " print(f\"Expected solution: {expected}\")\n", " print(f\"Filtered solution: {filtered[0].subgroups}\")\n", " print(f\"All solutions:{[sol.subgroups for sol in sols]}\")\n", " print(f\"-\"*100)\n", " \n", "print(\n", " \"Success rate of filter_polarity_contribution: \"\n", " f\"{success_polarity_contribution__Q__polar}/{count_multiple_needed}\"\n", " f\" ({success_polarity_contribution__Q__polar/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "4ee198be", "metadata": {}, "source": [ "#### filter_polarity_contribution — Q — apolar" ] }, { "cell_type": "code", "execution_count": 14, "id": "515115b7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_polarity_contribution: 10/1817 (0.55%)\n" ] } ], "source": [ "success_polarity_contribution__Q__apolar = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_polarity_contribution(sols, criteria=\"Q\", polarity=\"apolar\")\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions:{[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_polarity_contribution__Q__apolar += 1\n", "\n", " \n", "print(\n", " \"Success rate of filter_polarity_contribution: \"\n", " f\"{success_polarity_contribution__Q__apolar}/{count_multiple_needed}\"\n", " f\" ({success_polarity_contribution__Q__apolar/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "5ca47a1e", "metadata": {}, "source": [ "#### filter_polarity_contribution — R — polar" ] }, { "cell_type": "code", "execution_count": 15, "id": "c28ebd65", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_polarity_contribution: 1715/1817 (94.39%)\n" ] } ], "source": [ "success_polarity_contribution__R__polar = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_polarity_contribution(sols, criteria=\"R\", polarity=\"polar\")\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions:{[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_polarity_contribution__R__polar += 1\n", "\n", " \n", "print(\n", " \"Success rate of filter_polarity_contribution: \"\n", " f\"{success_polarity_contribution__R__polar}/{count_multiple_needed}\"\n", " f\" ({success_polarity_contribution__R__polar/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "7480c677", "metadata": {}, "source": [ "#### filter_polarity_contribution — R — apolar" ] }, { "cell_type": "code", "execution_count": 16, "id": "30d3d454", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success rate of filter_polarity_contribution: 143/1817 (7.87%)\n" ] } ], "source": [ "success_polarity_contribution__R__apolar = 0\n", "\n", "for smiles, expected, sols in zip(smiles_multiple_correct, expected_multiple_correct, store_multiple_solutions):\n", " filtered = unifac.filter_polarity_contribution(sols, criteria=\"R\", polarity=\"apolar\")\n", " \n", " # if len(filtered) > 1:\n", " # print(f\"SMILES: {smiles}\")\n", " # print(f\"Expected solution: {expected}\")\n", " # print(f\"All solutions:{[sol.subgroups for sol in sols]}\")\n", " # print(f\"-\"*100)\n", " # continue\n", " \n", " if expected == filtered[0].subgroups:\n", " success_polarity_contribution__R__apolar += 1\n", "\n", " \n", "print(\n", " \"Success rate of filter_polarity_contribution: \"\n", " f\"{success_polarity_contribution__R__apolar}/{count_multiple_needed}\"\n", " f\" ({success_polarity_contribution__R__apolar/count_multiple_needed*100:.2f}%)\"\n", ")\n" ] } ], "metadata": { "kernelspec": { "display_name": "ugropy", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.13" }, "nbsphinx": { "execute": "never" } }, "nbformat": 4, "nbformat_minor": 5 }