Pair programming with ChatGPT: A simple dice roller

Like many folks, I spent too much of the last couple days playing with the new release of ChatGPT. I’ve been trying discover what it’s good at, and how it breaks. At its best, it’s remarkable—I think it would actually pass many common “hiring screens” for programmers. And it has taken first place on an Advent of Code problem.

But there are various tricks which will break it. Programs with randomized output occasionally fool it, as do programs with multiple if branches. So I set down this morning for a short pair programming session, and wrote a classic dice roller with ChatGPT. The experience was fascinating. Things started out very frustrating, but they wound up pretty mind-blowing by the end.

Attempt 1: GPT fails hard at rand and sscanf in C

Let’s start with a simple statement of the problem:

Me: Write me a command-line program that takes D&D 5e dice expressions like “3d6+1”, rolls the dice specified, sums up the dice and any bonus modifier, and prints the result.

For example:

$ roll 3d6
12
$ roll 3d6
7
$ roll 1d10+5
11

ChatGPT:

#include<stdio.h>
#include<stdlib.h>
#include<time.h>introll_dice(intnum_dice,intnum_sides){intresult=0;for(inti=0;i<num_dice;i++){result+=rand()%num_sides+1;}returnresult;}intmain(intargc,char*argv[]){if(argc!=2){printf("Usage: roll <dice expression>\n");return1;}// Seed the random number generatorsrand(time(NULL));// Parse the dice expressionintnum_dice=0;intnum_sides=0;intbonus=0;sscanf(argv[1],"%dd%d+%d",&num_dice,&num_sides,&bonus);// Roll the dice and sum them upintresult=roll_dice(num_dice,num_sides)+bonus;printf("%d\n",result);return0;}

To compile and run this program, you can use the following commands:

gcc −o roll roll.c
./roll 3d6

This will compile the program and create an executable called roll, which you can then run with the 3d6 dice expression as an argument. This will roll three 6-sided dice and print the result. You can try different dice expressions to see the program in action.

Note: This is just an example program to illustrate how to roll dice in a D&D 5e dice expression. It is not intended to be a complete or fully-featured program.

There are some nice bits here—it actually seems to have some vague idea that 3d6 means “roll 3 dice with 6 sides each”. And it prints out a Usage notice and remembers to call srand. But the program is wrong, so let’s tell GPT about two “bugs”—one a real bug, and one that I was mistaken about.

Read more…