While
building a decentralized Twitter in a previous post, I included some code that implemented
JSON persistence. With the launch of Meta’s
Llama 3 this month, I thought it’d be a good opportunity to explore how a new LLM can help with coding.
Meta.AI is the chat frontend for Llama 3, which we will use after developing our persistence solution. As I write this (late April), Meta.AI is not available globally, so you may need to use a VPN to get to it. You may need to use a Facebook login for some things, like image generation, but that doesn’t seem to be a necessity.
First, I checked that I was actually talking to Llama 3, although this was not totally conclusive:
👁 Image
When looking at a
previous LLM (Llama 2), I asked it to do some long multiplication that it failed to manage. It did however “fail like a human.” Let us see how this model has progressed:
👁 Image
A quick, and much better-looking answer. Still wrong; the answer is
1,223,834,880. However, we aren’t here to worry about this known lacuna. Let’s return to the main topic: persistence with JSON.
Saving with JSON
JSON is a
portable data format, and while less efficient than an arbitrary native binary format, it can at least be checked by humans and implemented anywhere.
Because JSON can handle basic value types and some collections, we can in general map our instance data to JSON format. This is known as
marshalling and is a lot less of an issue today than it used to be. Whether you want to persist (i.e., save and load) user preferences, a turn in a game or just some plain old useful data, JSON has your back.
In this post I’ll use .NET and C#. While we can do just fine with the inbuilt System.Text.Json, the
Newtonsoft package is a bit nicer, so I’ll call that.
First I’ll set up a console project that will use the new style
top-level statements to cut out the need for
main statements and other boilerplate. So in a terminal:
👁 Image
I started with a quick Car class, which I wrote directly in Program.cs. We will use this as our target object:
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
public class Car
{
public enum CarColor {Red, Blue, White};
public string Brand;
public string Model;
public short MileageAtLastService;
public Car(string brand, string model, CarColor color)
{
this.Brand = brand;
this.Model = model;
this.MileageAtLastService = 0;
this.Color = color;
}
public void SetMileage(short mileage) => this.MileageAtLastService = mileage;
public override string ToString() => $"{Color} {Brand} {Model}" ;
}
The console output after running this is:
First car: Red Saab V90
To use the Newtonsoft.JSON package from the command line, we can just run this in the terminal:
JsonMarshalling> dotnet add package Newtonsoft.Json
You could also use the palette within VS Code, though I find that a little unreliable sometimes. So now that we have pushed the code for the car object to its own file, Car.cs, and namespace, CarClass, let’s do a simple conversion in Program.cs:
using CarClass;
using Newtonsoft.Json;
const string OUTFILE = @".cars.json";
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
string output = JsonConvert.SerializeObject(car);
Console.WriteLine($"Serialised: {output}");
The output to this in the terminal is:
Serialised: {"Brand":"Saab","Model":"V90","MileageAtLastService":0,"Color":0}
Oh, it didn’t work? But this is fine. The
enum used for CarColor needs to be handled separately as it isn’t a type recognized by JSON. We could write the conversion ourselves or just leave it to the existing converter by decorating the property in Cars.cs:
using Newtonsoft.Json
using Newtonsoft.Json.Converters;
...
[JsonConverter(typeof(StringEnumConverter))]
public CarColor Color {get; set;}
...
The output result is now correct:
Serialised: {"Brand":"Saab","Model":"V90","MileageAtLastService":0,"Color":"Red"}
What we want to work toward for a round trip is to persist a couple of cars down to a JSON file, restore, and then update one aspect (for instance, a garage service) and save back, finally checking the JSON on file. First, let’s check that we can save a list down to a file:
using CarClass;
using Newtonsoft.Json;
const string OUTFILE = @"./cars.json";
List<Car> cars = new List<Car>();
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
cars.Add(car);
car = new Car("Volkswagen", "Polo", Car.CarColor.White);
Console.WriteLine($"Uncle's car: {car}");
cars.Add(car);
string output = JsonConvert.SerializeObject(cars);
File.WriteAllText(OUTFILE, output);
The file produced is fine, containing two JSON objects:
👁 Image
Let’s generalize the persistence part and put it in its own project file:
using CarClass;
using Newtonsoft.Json;
namespace PersistenceServices
{
public static class Persistence
{
const string JSONFILE = @"./cars.json";
public static void WriteCarsToFile(List<Car> cars)
{
WriteToFile(cars);
}
public static List<Car> ReadCarsFromFile()
{
return ReadFromFile<Car>(JSONFILE);
}
private static void WriteToFile<T>(List<T> list)
{
string jsonString = JsonConvert.SerializeObject(list, Formatting.Indented);
File.WriteAllText(JSONFILE, jsonString);
}
private static List<T> ReadFromFile<T>(string filename)
{
List<T>? list = null;
if (File.Exists(filename))
{
string jsonString = File.ReadAllText(filename);
list = JsonConvert.DeserializeObject<List<T>>(jsonString);
}
else Console.WriteLine("No existing file found for " + filename);
return list ?? new List<T>();
}
}
}
Note that the two methods that actually call
JsonConvert do not need to know anything about the type being converted; they work with any generic list of types.
Here’s the round trip, which amends the second car’s service mileage:
using CarClass;
using Newtonsoft.Json;
using PersistenceServices;
List<Car> cars = new List<Car>();
Car car = new Car("Saab", "V90", Car.CarColor.Red);
Console.WriteLine($"First car: {car}");
cars.Add(car);
car = new Car("Volkswagen", "Polo", Car.CarColor.White);
Console.WriteLine($"Uncle's car: {car}"); cars.Add(car);
Persistence.WriteCarsToFile(cars);
List<Car> cars2 = Persistence.ReadCarsFromFile();
cars2.Last().MileageAtLastService = 2500;
Persistence.WriteCarsToFile(cars2);
The resulting file is even properly formatted:.
👁 Image
OK, so we created some persistence to a simple file using JSON, using a specific package. We ended up with these files in our project workspace:
👁 Image
Now let’s try to see how far we can get with Llama 3. Let’s break down what we did.
- Created a simple class to represent an owned car.
- Turned it into json.
- Used the json format to save and load into a local file.
So first let’s ask for a car class. The query is “Create a C# class that represents an owned car with these attributes: brand, make, miles since last service and color.”
👁 Image
Great. It also gave examples of using the class and setting the properties. Let us see if we can persuade it to make the color property an
enum:
👁 Image
It did! Even with my spelling mistake in the request!
Now let’s see if it can use JSON to convert. Let’s force it to use Newtonsoft.json too.
👁 Image
It thinks this will be the output:
👁 Image
Of course, we already know this won’t work as it is. We need to fix the enum. We could try to tell Llama this, but we only know it isn’t correct because we’ve been here before. However, let’s take the win.
The good news is that Llama 3 was quick, well formatted and good with basic requirements.
- It allowed us to work on the code in a continuous fashion over time in one thread.
- It understood how to use an arbitrary named package.
- It correctly created example instances of Car, as well as some nice car colors.
- The answers are nicely formatted.
Where it went wrong was a detail that I happened to have worked through in advance. This leaves us in roughly the same place we’ve been in all the time with
LLM coding: It can quickly create a working template that we can tailor, but it will have inaccuracies that will require fixing. This means it is
a useful bootstrapping tool for a working developer but not yet a sort of magic solution maker.
