Commit 0e924ae
FEAT: Threshold and ratio configuration and testing file for optimal threshold and ratio configuration (#21)
* improved systems
* FIX: Refactor negative_to_positive_ratio parameter to be optional in SentinelLocalIndex.
FEAT: created a testing tool for best threshold and ratio analysis
* DOCS: Add section on testing optimal thresholds and data ratios in README
* FEAT: Added a review mode for detailed information regarding each case with optional flags.
DOCS: Updated relevent documentation with these fixes
* FEAT: Enhance threshold testing with performance metrics and results-only mode
* FEAT: Set default negative_to_positive_ratio to 5.0 and add error handling and fallback for ratio adjustments
* FIX: Update user profile creation to exclude sexual content examples
Feat: Adjusts message metrics dynamically
* FIX: Comment out sexual content examples in user profile creation to prevent inclusion
* FEAT: Added caching and caching tests to the main model load, reduced load time of model exponentially after the first caching.
TESTS: Updated embedding tests to include caching and its management functions
FEAT: Updated the example script for testing purposes to include caching mechanics
* DOCS: Fixed formatting for better PEP8 format
* STYLE: Apply PEP 8 formatting and update examples
- Apply PEP 8 formatting to Example_Threshold_Script.py
- Update embeddings.safetensors
- Update sentinel_against_hate.ipynb
- Fixed line length violations (max 79 characters)
- Corrected indentation and spacing
- Enhanced readability while maintaining functionality
* FEAT: Add caching option to SentinelLocalIndex for improved load performance
* REFACTOR: Refactor user profile creation to use external test data for speech examples
* TEST: Update mean_of_positives tests to return 0.0 for negative and empty score arrays, fixes edge case, NaN returns.
* CHORE: Remove redundant Cache_Model variable in `calculate_rare_class_affinity`, update example file to use path/to/index rather than local path
* TEST: Add edge case tests for aggregation functions and contrastive_components in score_formulae and SentinelLocalIndex
* Chore: removed redundant imports
* chore: Changed caching to be false by default, to preserve old functionality, removed redundant exports, added no-cache flag to the testing script
* All requested changes made
* fix capitalisation issue
* The should fix thee build issues for CI tests as pandas ^1.0.0 didn't have support for PEP 517 builds hence swapped to ^2.0.0 which is compatable - may require further testing however didn't impact functionality of code
---------
Co-authored-by: Vladislav Kondratyev <chikimoni61@gmail.com>1 parent e80915d commit 0e924ae
14 files changed
Lines changed: 1783 additions & 76 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
14 | 30 | | |
15 | 31 | | |
16 | 32 | | |
| |||
65 | 81 | | |
66 | 82 | | |
67 | 83 | | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
68 | 94 | | |
69 | 95 | | |
70 | 96 | | |
| |||
109 | 135 | | |
110 | 136 | | |
111 | 137 | | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
112 | 169 | | |
113 | 170 | | |
114 | 171 | | |
| |||
0 commit comments