Enhance aider's Markdown editing capabilities with --watch-files

[lockmeister]

Aider is an excellent context-management tool for AI. It would be powerful to extend this to text editing other than programming. For example, Ive used aider --watch-files on a folder of markdown files which is also opened as a vault using obsidian.md.

Currently, aider's --watch-files functionality is primarily optimized for code editing. When used with Markdown files, it misinterprets text-based prompts (especially those containing "AI!") as comment removal requests, rather than generating new content. This issue aims to enhance aider's ability to handle Markdown files effectively, allowing users to leverage its context management, file handling, and chat capabilities for writing and editing text in Markdown, while still using Git
for version control. The goal is to achieve this without degrading aider's core code editing abilities.

Desired Outcome:

• aider should correctly interpret text-based prompts in Markdown files, generating new content when requested.
• aider should not misinterpret "AI!" as a comment when it's part of a sentence in a Markdown file.
• aider should respect Markdown syntax when making edits.
• Users should be able to leverage aider's context management, file handling, and chat capabilities for Markdown files.
• aider's core code editing capabilities should remain unaffected.
• Users should have the option to enable a markdown-specific prompt for more tailored behavior.

Proposed Solution:

The following changes will be implemented to address the issue:

1 Flexible File Type Detection:
• Implementation: Modify FileWatcher.filter_func in aider/watch.py to process all files with AI comments, regardless of file extension.
• Purpose: This allows aider to be used with any file type, including Markdown, while still using the AI comment syntax.
2 Context-Aware Comment Detection:
• Implementation: Keep the context-aware comment detection in FileWatcher.get_ai_comments in aider/watch.py, but ensure the ai_comment_pattern is specific enough
to avoid misinterpreting "AI!" as a comment when it's part of a sentence.
• Purpose: This prevents aider from incorrectly removing "AI!" from text.
3 Optional Markdown-Specific Prompt:
• Implementation:
• Keep the markdown_prompt in CoderPrompts in aider/coders/base_prompts.py.
• Add a new command-line argument --use-markdown-prompt to aider/args.py.
• Modify Coder.format_chat_chunks in aider/coders/base_coder.py to use the markdown_prompt if --use-markdown-prompt is enabled.
• Purpose: This allows users to choose whether to use a prompt specifically tailored for Markdown files.
4 Text Generation Logic:
• Implementation: Keep the text generation logic in FileWatcher.process_changes in aider/watch.py with the watch_code_prompt and watch_ask_prompt prompts.
• Purpose: These prompts will be used when the user ends a comment with AI! or AI? respectively.

Implementation Steps:

1 Modify aider/watch.py:
• Update filter_func to process all files with AI comments.
• Keep the ai_comment_pattern.
• Keep the process_changes method.
2 Modify aider/coders/base_prompts.py:
• Keep the markdown_prompt.
3 Modify aider/coders/base_coder.py:
• Update format_chat_chunks to use the markdown_prompt if --use-markdown-prompt is enabled.
4 Modify aider/args.py:
• Add the --use-markdown-prompt command-line argument.

Testing:

• Thoroughly test aider with both code files and Markdown files to ensure that the changes are working correctly and that no regressions have been introduced.
• Test with and without the --use-markdown-prompt option.

Additional Notes:

• This issue addresses the specific problem of using aider with Markdown files and the AI! syntax.
• The proposed solution aims to be flexible and maintainable.
• The changes should be well-documented.

Aider version: 0.69.1
Python version: 3.12.3
Platform: Linux-6.8.0-51-generic-x86_64-with-glibc2.39
Python implementation: CPython
Virtual environment: Yes
OS: Linux 6.8.0-51-generic (64bit)
Git version: git version 2.43.0

[turian]

Agreed. Editing MARKDOWN-only repos is one thing I commonly do with markdown. e.g. when I'm doing research

[paul-gauthier]

Thanks for trying aider and filing this issue.

Watch files should already work with markdown? Are you having a specific problem?

[turian]

@paul-gauthier According to the documentation, it only works with specific comments starts ('#' and '//'). These don't necessarily play well with Markdown.

@lockmeister might weigh in more specifically. Their issue was well thought out but maybe they can say what their highest-priority changes would be

[paul-gauthier]

Ya, just use them like they are comments. Aider should respond to them and remove them.

[lockmeister]

OK, I'm really impressed @paul-gauthier .
When I first raised this issue, there were some problems with -watch-files in markdown.
However it seems to work well now.

In my screen recording, you can see aider on the left and obsidian.md on the right. Both are open in the same demo folder with markdown notes. In the screen recording, I demo

• adding markdown notes from the aider command line
• editing notes (conversion to bullet points) using #AI! inline comment
• add mermaid diagram using #AI! inline comment
• add a code block usin g#AI inline comment.

I think that aider will be a very useful tool for note taking with obsidian, because it allows good context management through adding and dropping related notes, /web scraping, plus its general understanding of things such as mermaid and markdown formatting.

I'm not sure what fixed the problems; I'm now using deepseek instead of sonnet 3.5. Also perhaps my dummy repo for this test may be different to my personal environment (dummy environment much simpler).

@turian @paul-gauthier , now that this is working I'm pretty happy as-is. I'll think about any additional features, perhaps a command that understands [[markdown links]]. E.g. so you can easily /add all the pages linked in the current page.
I also think a /youtube command that added the transcript of a yt video would be handy. I've written my own scripts for this, however for some videos I have to use whisper to do the transcription myself locally.

Overall, @paul-gauthier I continue to be highly impressed with aider, thanks for all your work. Let me know if I can help you with anything.