Developing gmi2md with Claude

Contents

Motivation

I wanted a gemtext-to-Markdown translator for my SDF homepage. There, I adopted a survey of “small-Internet” protocols by zzo38 from plain text to gemtext. Gemtext is a simple line-oriented hypertext format created for the Gemini protocol. My wish was to generate an HTML version of the gemtext automatically. Pandoc didn’t support gemtext. A converter I found on GitHub didn’t convert lists correctly and lacked a license. I had recently registered for Claude, an AI assistant developed by Anthropic, which gave me the idea to try it.

Development

The initial version of gmi2md was developed by Claude 3.5 Sonnet in 14 human-guided steps in a single chat. I have published the chat log in case others find it informative. While I had used ChatGPT (3.5, 4, and 4o) for code before this project, it had mostly been to translate between languages (often shell script to Python), create scaffolding, and refactor. When I generated entire programs from scratch, they were trivial. This was also my first time generating code with Claude.

I began by asking Claude if it knew about gemtext. It did, and briefly explained the syntax. Next, I asked it to compare gemtext with Markdown. My idea was to create a concrete reference for Claude to work off later. I prompted it to gradually improve the code until I was satisfied with the code’s functionality and shape for the initial commit. At one point I made manual edits to the code and told Claude to use my edited version.

Dan Shipper 📧 (@danshipper, 2024-07-30):
“Friends don’t let friends use Claude without saying “make it better” at least 3x after every prompt”

Using Claude felt similar to collaborating with a human in that I addressed it in plain text in a simulated chat and different in that I employed specific tricks. It probably helped to create a summary upfront. It really was productive to ask Claude to “make it better” at several points. (A human collaborator would not appreciate a vague and onerous request like this.) The effect was fairly dramatic the first time, when Claude applied modern PHP 8 practices to the code it had just generated, getting rid of PHPDoc types, replacing strpos with str_contains and str_starts_with, and using match. It also unnecessarily replaced variable substitution with sprintf.

The code the AI delivered at the end of the chat worked well enough to be useful verbatim, which surprised me. I had learned to expect AI code to be somewhat broken even in basic usage. However, my first impression of the non-broken code turned out to be too optimistic. The code required more bugfixes and stumbled on more edge cases than I had thought. Claude also misremembered the gemtext syntax, which took me some time to notice. It thought a space was required after the leading # characters in a heading.

The generated code was at first mildly object-oriented with a single GemtextToMarkdownConverter class that performed the conversion. In the process of getting the converter to output Markdown that corresponded to the gemtext, Claude introduced mutable state to the class, which I asked it to remove. Later, after making several commits myself, I noticed a repeating pattern in the code that could be refactored. In two places an enum designating the gemtext line type was matched on to choose an action. You could replace the enum with classes for the individual line types, then call their methods—exactly what object-oriented programming is good for. I asked Claude to refactor the code to a more object-oriented design in order to remove the redundancy. It succeeded in a single step, creating the classes and an interface they formed to.

Conclusions

The key takeaway is perhaps that an AI assistant is not going to create what you want after your opening prompt. When the AI creates something flawed that could be a starting point for what you want, keep going and write prompts to improve it in broad and in specific ways. Too many programmers apparently try to one-shot a solution to their problem by rewriting the opening prompt. They should be iterating.

I have wasted half an hour repeatedly clicking “retry” for a tiny Python snippet. It is frustrating to say the least and might be slower than writing it myself. The user experience is terrible.

(This is a quote from a real comment paraphrased for anonymity.)

PHP is not a language I write very often. I also don’t keep up with the latest developments. The AI helped me write a program in PHP faster and with better idioms than I would have on my own. (I determined experimentally that the modern code Claude created required PHP 8.2. When I removed readonly from the main class, it worked in 8.0.) The AI also correctly performed a pretty complex refactor with only a prose description of how I wanted it to go. Overall, I was happy with the experience and the result.