llm programming

 have recently been doing a lot of llm programming and encountering a common situation

 

llm will often suggest solutions that are new aspects of python or libraries i haven't encountered before, using a different approach than i usually would

llm produces the boilerplate code and of course there's something wrong with it, or, later i need to go back and update this code that i didn't write, so i end up going through a series of questions to determine what each of these new functions or methods do.

---

when i initially started developing again in python, i decided to write a simple sftp client as a learning project.  i started using chatgpt 3 which was what there was at the time, and i just had it write individual functions as i plugged along writing my program.  oh i need a function to connect to a sftp site, what does that return, how do i grab a file, etc stuff like that

i got a basic working graphical client out of all that and decided to keep adding features to it, and as i did so the llm stuff advanced, things like claude appeared and you could feed an entire like module into it and be like, what do you think of this?

as the llms got more advanced, i started taking more experimental approaches -- the llm would suggest some kind of thread pool and dispatcher model that i would never, being an ancient MS-DOS dipshit, think of, and i'd be like, ok let's see how this goes.  

it would produce the code and i would go look at it and learn how it actually worked and start refining these new capabilities to do more things as i thought of them.  

then programs like Aider came out, now there's claude code, roo code, qwen code, etc, essentially agents that analyze your code and then handle all the behind the scenes stuff of co-ordinating what needs to be done where when you say something like 'convert this to pyqt6'

so, i let aider do some relatively large code changes like, maybe i had made a mess of error handling throughout the app and wanted to try to standardize and centralize it, it would analyze the entire program and find all the spots where errors were displayed and then rewrite them all according to some agreed upon standard

however, as the codebase started to grow, the modules got so large that no llm model i could find could fit an entire model into its context window, and if it could, that still left the issue of bugs that occur across module boundaries (which is like, all of them at this point)

along comes roo code with its content indexing.  

so here's the deal, go get vs code, install the roo code extension, set up some kinda local llm host like vllm or hugging face or whatever locally and run mxbai or gte-large text embedding models on it (it is absolutely crucial you use an excellent embedding model or you will get total shit), then set up a qdrant vector database for it to use.  then, use a GOOD coding model, i used qwen-3-code, its basically the best CHEAP/affordable model out there.

roo breaks up your code into indexed sections so it can quickly locate all the related code for the operation you asked for without stuffing the entirety of the application into each api request

it ALSO has an Architect mode that breaks up your plain-english-statement into a very granular detailed series of simple steps that take place sequentially and in their own contexts

so i'm now able to make major design modifications across multiple modules, fix bugs across multiple modules, etc, even in a situation where each module is thousands of lines of code.  it's fucking awesome. the big example is after like a week of experimenting with different embeddings, i got it to convert the entire codebase from pyqt5 to pyqt6, flawlessly.  i mean that, really, like, i didn't have to do shit, it RAN immediately after all the changes were made.  amazing.