**This pull request will need a separate migration pull-request**
I have added an alert box in two places when the user tries to predict with stereo chemistry.
When a user predicts a pathway with stereo chemistry an alert box is shown in that node's hover.
To do this I added two new fields. Pathway now has a "predicted" BooleanField indicating whether it was predicted or not. It is set to True if the pathway mode for prediction is "predict" or "incremental" and False if it is "build". I think it is a flag that could be useful in the future, perhaps for analysing how many predicted pathways are in enviPath?
Node now has a `stereo_removed` BooleanField which is set to True if the Node's parent Pathways has "predicted" as true and the node SMILES has stereochemistry.
<img width="500" alt="{927AC9FF-DBC9-4A19-9E6E-0EDD3B08C7AC}.png" src="attachments/69ea29bc-c2d2-4cd2-8e98-aae5c5737f69">
When a user does a prediction on a model's page it shows at the top of the list. This did not require any new fields as the entered SMILES does not get saved anywhere.
<img width="500" alt="{BED66F12-5F07-419E-AAA6-FE1FE5B4F266}.png" src="attachments/5fcc3a9b-4d1a-4e48-acac-76b7571f6507">
I think the alert box is an alright solution but if you have a great idea for something that looks/fits better please change it or let me know.
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#250
Co-authored-by: Liam Brydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: Liam Brydon <lbry121@aucklanduni.ac.nz>
Fixes:
- Register now works again with the html form action pointing to `register` instead of `login`
Since this is a major issue the above change should probably be merged soon. However, I will open another issue (#248) suggesting we add better help for password creation as currently we give password requirements but do not check them.
Reviewed-on: enviPath/enviPy#247
Reviewed-by: Tobias O <tobias.olenyi@envipath.com>
Co-authored-by: Liam Brydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: Liam Brydon <lbry121@aucklanduni.ac.nz>
Reactant Filter SMARTS as well as Product Filter SMARTS are now reflected when applying rules.
Fixes#245
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#246
This PR moves all the collection pages into the new UI in a rough push.
I did not put the same amount of care into these as into search, index, and predict.
## Major changes
- All modals are now migrated to a state based alpine.js implementation.
- jQuery is no longer present in the base layout; ajax is replace by native fetch api
- most of the pps.js is now obsolte (as I understand it; the code is not referenced any more @jebus please double check)
- in-memory pagination for large result lists (set to 50; we can make that configurable later; performance degrades at around 1k) stukk a bit rough tracked in #235
## Minor things
- Sarch and index also use alpine now
- The loading spinner is now CSS animated (not sure if it currently gets correctly called)
## Not done
- Ihave not even cheked the admin pages. Not sure If these need migrations
- The temporary migration pages still use the old template. Not sure what is supposed to happen with those? @jebus
## What I did to test
- opend all pages in browse, and user ; plus all pages reachable from there.
- Interacted and tested the functionality of each modal superfically with exception of the API key modal (no functional test).
---
This PR is massive sorry for that; just did not want to push half-brokenn state.
@jebus@liambrydon I would be glad if you could click around and try to break it :)
Finally closes#133
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#236
Co-authored-by: Tobias O <tobias.olenyi@envipath.com>
Co-committed-by: Tobias O <tobias.olenyi@envipath.com>
I added playwright for frontend testing and got a couple simple test cases working.
I have updated pyproject.toml but it can also be installed with `pip install pytest-playwright` followed by `playwright install`
With the django server running you can do `playwright codegen http://localhost:8000/` which will generate test code based on the actions you take on the webpage it opens. Be sure to change the target to pytest in the code pop up.
I will add more test cases but @jebus and @t03i feel free to add more. Especially once we are done with the full front-end redesign.
I have put the tests under `tests/frontend/` but I am not sure how to add them to the CI. They give steps for CI integration but maybe we want to somehow include them in our exisiting CI yaml? https://playwright.dev/python/docs/ci-intro
Reviewed-on: enviPath/enviPy#218
Reviewed-by: Tobias O <tobias.olenyi@envipath.com>
Co-authored-by: Liam Brydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: Liam Brydon <lbry121@aucklanduni.ac.nz>
This will hack the ketcher submission to work again (see #207).
The problem seems to be that the iframe loads slower than the script tag so the reference is not available on page load.
Registering from within the code to poll until ketcher is ready is a bit messy.
Tracked the introduced dept in #212.
Reviewed-on: enviPath/enviPy#213
Co-authored-by: Tobias O <tobias.olenyi@envipath.com>
Co-committed-by: Tobias O <tobias.olenyi@envipath.com>
Adds packages to the footer (instead of Browse link) and removes search from footer (search page not directly linked anymore).
Packages are also available in Browse now.
This is temporary until there is a proper data browsing page.
Co-authored-by: jebus <lorsbach@envipath.com>
Reviewed-on: enviPath/enviPy#196
Co-authored-by: Tobias O <tobias.olenyi@envipath.com>
Co-committed-by: Tobias O <tobias.olenyi@envipath.com>
## Major Changes
- Predict Pathway is now a separate view as it is meant as adavanced prediction interface
## Current status
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#188
Co-authored-by: Tobias O <tobias.olenyi@envipath.com>
Co-committed-by: Tobias O <tobias.olenyi@envipath.com>
Implementing a search modal (stretching the level of dynamic that is possible without going to frameworks).
## Major Change
- Search needs packages and is available everywhere now; so had to add reviewed and user packages to global context.
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#185
Co-authored-by: Tobias O <tobias.olenyi@envipath.com>
Co-committed-by: Tobias O <tobias.olenyi@envipath.com>
This PR introduces an overhaul for the front page and login features while keeping the rest of the application intact.
## Major Changes
- TailwindCSS + DaisyUI Integration: Add modern CSS framework for component-based utility styling
- Build System: Added pnpm for CSS building; can be extended for updated frontend builds in the future
- Navbar + Footer: Redesigned and includable; old version retained for unstyled elements
- Optimized Assets: Added minified and CSS-stylable logos
## New Features
- Static Pages: Added comprehensive mockups of static pages (legal, privacy policy, terms of use, contact, etc.). **Note:** These have to be fixed before a public release, as their content is largely unreviewed and incorrect. Probably best to do in a separate PR that only contains updates to these.
- Discourse API: Implement minimal features based on RestAPI for controllable results.
## Current bugs
- [x] The static pages include the default navbar and footer on the login page. This will likely not work, as users need to access it before logging in; no good workaround so far (problem with Django templating system).
- [ ] The front page predict link is currently non-functional; the redesigned page is almost ready but better done in a separate PR as it also touches Django code.
- [x] Visual bug with the news cards. Still intend to fix for this PR
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#179
Reviewed-by: jebus <lorsbach@envipath.com>
Co-authored-by: Tobias O <tobias.olenyi@envipath.com>
Co-committed-by: Tobias O <tobias.olenyi@envipath.com>
## Changes
- All text input fields are now cleaned with nh3 to remove html tags. We allow certain html tags under `settings.py/ALLOWED_HTML_TAGS` so we can easily update the tags we allow in the future.
- All names and descriptions now use the template tag `nh_safe` in all html files.
- Usernames and emails are a small exception and are not allowed any html tags
Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Co-authored-by: jebus <lorsbach@envipath.com>
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#171
Reviewed-by: jebus <lorsbach@envipath.com>
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
# Summary
I have introduced a new base `class Dataset` in `ml.py` which all datasets should subclass. It stores the dataset as a polars DataFrame with the column names and number of columns determined by the subclass. It implements generic methods such as `add_row`, `at`, `limit` and dataset saving. It also details abstract methods required by the subclasses. These include `X`, `y` and `generate_dataset`.
There are two subclasses that currently exist. `RuleBasedDataset` for the MLRR models and `EnviFormerDataset` for the enviFormer models.
# Old Dataset to New RuleBasedDataset Functionality Translation
- [x] \_\_init\_\_
- self.columns and self.num_labels moved to base Dataset class
- self.data moved to base class with name self.df along with initialising from list or from another DataFrame
- struct_features, triggered and observed remain the same
- [x] \_block\_indices
- function moved to base Dataset class
- [x] structure_id
- stays in RuleBasedDataset, now requires an index for the row of interest
- [x] add_row
- moved to base Dataset class, now calls add_rows so one or more rows can be added at a time
- [x] times_triggered
- stays in RuleBasedDataset, now does a look up using polars df.filter
- [x] struct_features (see init)
- [x] triggered (see init)
- [x] observed (see init)
- [x] at
- removed in favour of indexing with getitem
- [x] limit
- removed in favour of indexing with getitem
- [x] classification_dataset
- stays in RuleBasedDataset, largely the same just with new dataset construction using add_rows
- [x] generate_dataset
- stays in RuleBasedDataset, largely the same just with new dataset construction using add_rows
- [x] X
- moved to base Dataset as @abstract_method, RuleBasedDataset implementation functionally the same but uses polars
- [x] trig
- stays in RuleBasedDataset, functionally the same but uses polars
- [x] y
- moved to base Dataset as @abstract_method, RuleBasedDataset implementation functionally the same but uses polars
- [x] \_\_get_item\_\_
- moved to base dataset, now passes item to the dataframe for polars to handle
- [x] to_arff
- stays in RuleBasedDataset, functionally the same but uses polars
- [x] \_\_repr\_\_
- moved to base dataset
- [x] \_\_iter\_\_
- moved to base Dataset, now uses polars iter_rows
# Base Dataset class Features
The following functions are available in the base Dataset class
- init - Create the dataset from a list of columns and data in format list of list. Or can create a dataset from a polars Dataframe, this is essential for recreating itself during indexing. Can create an empty dataset by just passing column names.
- add_rows - Add rows to the Dataset, we check that the new data length is the same but it is presumed that the column order matches the existing dataframe
- add_row - Add one row, see add_rows
- block_indices - Returns the column indices that start with the given prefix
- columns - Property, returns dataframe.columns
- shape - Property, returns dataframe.shape
- X - Abstract method to be implemented by the subclasses, it should represent the input to a ML model
- y - Abstract method to be implemented by the subclasses, it should represent the target for a ML model
- generate_dataset - Abstract and static method to be implemented by the subclasses, should return an initialised subclass of Dataset
- iter - returns the iterable from dataframe.iter_rows()
- getitem - passes the item argument to the dataframe. If the result of indexing the dataframe is another dataframe, the new dataframe is packaged into a new Dataset of the same subclass. If the result of indexing is something else (int, float, polar Series) return the result.
- save - Pickle and save the dataframe to the given path
- load - Static method to load the dataset from the given path
- to_numpy - returns the dataframe as a numpy array. Required for compatibility with training of the ECC model
- repr - return a representation of the dataset
- len - return the length of the dataframe
- iter_rows - Return dataframe.iterrows with arguments passed through. Mainly used to get the named iterable which returns rows of the dataframe as dict of column names: column values instead of tuple of column values.
- filter - pass to dataframe.filter and recreates self with the result
- select - pass to dataframe.select and recreates self with the result
- with_columns - pass to dataframe.with_columns and recreates self with the result
- sort - pass to dataframe.sort and recreates self with the result
- item - pass to dataframe.item
- fill_nan - fill the dataframe nan's with value
- height - Property, returns the height (number of rows) of the dataframe
- [x] App domain
- [x] MACCS alternatives
Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#184
Reviewed-by: jebus <lorsbach@envipath.com>
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
## Changes
- Ability to change the threshold from a command line argument.
- Names of data packages included in model name
- Names of data, rule and eval packages included in the model description
- EnviFormer models are now viewable on the admin site
- Ignore CO2 for training and evaluating EnviFormer
Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#173
Reviewed-by: jebus <lorsbach@envipath.com>
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
The caching is now finished. The cache is created in `settings.py` giving us the most flexibility for using it in the future.
The cache is currently updated/accessed by `tasks.py/get_ml_model` which can be called from whatever task needs to access ml models in this way (currently, `predict` and `predict_simple`).
This implementation currently caches all ml models including the relative reasoning. If we don't want this and only want to cache enviFormer, i can change it to that. However, I don't think there is a harm in having the other models be cached as well.
Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#156
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
liambrydon
t03i
