Notebook template for transforms to run on Google Colab #851
+268
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shahrokhDaijavad
approved these changes
Dec 3, 2024
There was a problem hiding this comment.
Thanks, @Ryan-Gordon-314159. As we discussed, the open colab icon will work only when this PR is merged. Other than that, I have tested this by manually running it on Google Colab.
touma-I
requested changes
Dec 3, 2024
There was a problem hiding this comment.
We need to change the notebook to tell a story that would be interesting to the end user. What do you think of this?
- Step 1: Enable Collab
- Step 2: Pip install
- Step 3: Identify interesting pdf files and use web crawler to fetch them. If possible, identify pdf contant that has OCR in it
- Step 4: Configure the transform to generate MD, run the transform
- Step 5: Show output from the transform, not only the new content but any other fields that the transfrom may have produced that may be of interest based on the content
- Step 6: Repeat step 4 and 5 above but with a different configuration of the tansform for example, generating Json instead of MD or extracting OCR, etc.
There was a problem hiding this comment.
We want to consider getting rid of my_utils.py file, if possible.
There was a problem hiding this comment.
please do not download my_utils.py. I won't be able to merge this PR with this in it.
touma-I
requested changes
Dec 19, 2024
There was a problem hiding this comment.
please do not download my_utils.py. I won't be able to merge this PR with this in it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
This PR contains a notebook that can serve as a guide for porting transforms over to work on Google Colab
Related issue number (if any).
Issue #844