fix(NegativeSampleGenerator): Prevent duplicate user IDs when getting unique IPs #958
Conversation
|
Hello there, We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process. Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6 Thank you for contributing to Nextcloud and we hope to hear from you soon! (If you believe you should not receive this message, you can add yourself to the blocklist.) |
|
Is there anything major blocking this from being merged? I've tested out these changes for myself, and it seems to fix #860. |
… unique IPs Signed-off-by: David Dreschner <[email protected]>
Signed-off-by: David Dreschner <[email protected]>
Signed-off-by: David Dreschner <[email protected]>
Signed-off-by: David Dreschner <[email protected]>
ChristophWurst
force-pushed
the
fix/unique-datasets-only
branch
from
9690f4f to
0b71fb1
Compare
|
@ChristophWurst : You're welcome! I'm looking forward to the release of my changes. |
To weight the usage of IPs in the data model, the user id is being added multiple times to the dataset. The exact number is being determined here:
suspicious_login/lib/Service/DataLoader.php
Line 80 in 9043a66
The problem is that this isn't always being taken into account when generating the training data. In case the dataset contains no IP with exactly one user id in the array, the array is malformed by containing only user ids (16 columns in size) in this variable:
suspicious_login/lib/Service/DataLoader.php
Line 107 in 9043a66
The reason is the code in the
getUniqueIPsPerUserfunction. It returns only IPs with exactly one user id in the second dimension, while not checking if the user id was already added to the IP during the previous data transformation.The Rubix ML library then throws an
InvalidArgumentExceptiondue to the unequal number of columns when trying to merge the malformed array with the other datasets here:suspicious_login/lib/Service/DataLoader.php
Line 115 in 9043a66
This PR will fix the issue and adds an unit test to prevent it during future refactoring steps.
Closes #860
Closes #933