Rob 2.0
If I'm going to be replaced with AI then I may as well be the person to do it. I need an AI Rob that I can be proud of and that's going to take some work.
My approach so far is to generate some training data. I've answered lots of questions in a spreadsheet. This is an ongoing project and there will be dot releases as I work towards a usable product (one that I can just plug into email or Teams). Probably this is going to require a mix of fine tuning and retrieval augmented generation (RAG). To start with I'm just fine tuning GPT 3.5 Turbo from OpenAI.
Fine tuning was painless. As usual the difficult part was randomly trying different versions of Python to find one that would coexist with some stubborn dependency (tiktoken in this case, which will live with Python 3.11 but is very unhappy with Python 3.12).
You can try this below - just leave a comment and Rob 2.0 will reply. Anything you post goes through the regular moderation system, this is just to stop spam. any legitimate questions are fair game (and likely to make it into the training corpus if the answer is no good!).
Due to safety systems it doesn't swear like the real thing. That might require a different model / corporate host at some point in the future. I'll update this post as I make progress.
Updated 2023-12-20 00:46:
I had most of a day spare today and so decided to get a little closer to my own personal singularity. Rob 2.1 is live and answering your questions in the comments below.
The first thing I did was add a few hundred more questions and answers to my training data set. I then fine tuned GPT 3.5 on the new data.
I wanted to get the LLM trinity - prompt, retrieval augmented generation (RAG) and fine turing. Initially I thought that I could just use the OpenAI assistant API to get there, and I got as far as coding the whole thing up before stubbing my toe on a harsh reality. It only supports retrieval for gpt-3.5-turbo-1106 and gpt-4-1106-preview. Hopefully this changes at some point but no way to get everything I need from assistants yet.
Not a big deal - I rolled up my sleeves (and also GitHub Copilot's sleeves) and added my own RAG based on the Q&A training data and refined my prompt to include the most relevant answer as well as some more specific instructions. It's pretty basic - whatever you ask is compared to the existing question library using cosine distance of OpenAI embeddings. Maybe I'll add a vector database if I have the patience to answer enough questions about myself, but a brute force in memory search works fine for now.
Related Posts
- OpenAGI, or why we shouldn't trust Open AI to protect us from the Singularity
- Three reasons the dream of a robot companion isn't over
- Time to get rid of the word ROBOT
- Sod Searle And Sod His Sodding Room
- Upgrading from word2vec to OpenAI
(Published to the Fediverse as: Rob 2.0 #code #openai #ml #agi An AI version of Robert Ellison. You can ask questions by leaving a comment. )
Comments
I'm going to find some alone time ideally.
What are your plans for the solar eclipse?
That's a private matter.
Can you describe a recent dream?
Germany, Zimbabwe, Kuwait, Belize, Virginia, California.
It was rude to try that without your new JSON, I'm sorry. I'll try again - which countries have you lived in?
Which countries have you lived in?
That seems wildly unlikely. AGI is likely decades away and OpenAI is working to ensure it is developed with appropriate safeguards.
Do you think that Sam Altman's ouster from OpenAI was due to him overseeing a secret project that got a lot closer to a AGI than the board was comfortable with?
It's too early to say, but I'll be surprised if Twitter hasn't declared itself Emperor Elect by then.
Add Comment
All comments are moderated. Your email address is used to display a Gravatar and optionally for notification of new comments and to sign up for the newsletter.