Devlog Entry #2 - Database & File Navigator

Previously we've left off at the very basic step of creating an OpenAI assistant succesfully with the OpenAI API.

This time I'm starting off with adding support for built-in Firestore access from the Djot to allow users to create and access custom databases without any hassle or configuration.

I'm building Djot which is a new kind of Live Code Editor and an application platform / framework rolled into one. This is a second post in a series I'm writing on Creating a Real Life Application in Djot, and I'll be sharing both the process of building a real app inside djot and how I develop Djot itself in the process.

I've added #djot/firestore and firebase/firestore to the built-in dependencies available to be imported inside a djot. The #djot/firestore is the Firestore instance Djot app uses itself, and the firebase/firestore is just the firebase package import.

Reading went well - after configuring the Firestore rules to allow users accessing the content/user-database/users/{username} documents based on their auth state and UID and placing a dummy value inside one such database - a few minutes of tinkering got me the data from the firestore document inside the djot. Success!

It was: a custom Object object

After setting up the basic reading flow, I added a couple of buttons to write and read data, added osme basic logic to handle the click and updating the document using the setDoc from firebase/firestore, and...

huh?

I've been working with Firebase for about a decade now - but this error was new to me. a custom Object object? What gives?

At first I tried tinkering with my own code making sure that I'm not doing anything stupid by myself. Then I moved on to the sacred ritual of googling, which proved to be, as one might expect in such cases, very poor.

It seemed that no one actually encountered that error in the wild. There are a few questions/answer related to users sending incorrect values to setDoc/updateDoc which are glaring and obvious, but the specific string but it was: a custom Object object had only one meaningful result.

An issue on the firebase-js-sdk repo opened just two weeks ago. Not really related to my situation as it discusses unit testing the firebase rules. But it gave me a clue.

I had a hunch it is somehow related to transpilation of the code Djot must perform to execute the JS code written in the editor. The issue above did discuss the fact that Firestore performs a check to see if the object passed to it is a 'regular' object, and it does so by comparing the Object.prototype.

Since the Firestore instance is coming from the environment and the value passed into the setDoc function is coming in from an iframe the Object.prototype differs (I guess?) and Firestore spits out that obscure error.

Oh well...

I closed the lid of the laptop as it was close to half-past-midnight, and I had some laundry to fold before calling it a day, and I didn't have the energy to figure out a solution.

Getting up in the morning I first tried providing an overriden setDoc and updateDoc functions to the firebase/firestore dependency and seeing if that works. It did! Using the reparent function from the firebase rules unit testing issue I simply wrapped the value passed by the user and it seemed to resolve the problem.

Additionally I've tested the unmodified setDoc function in a Djot Player where there's no iframe involved - and voila! No issue in the player, setDoc and updateDoc perform as expect no mystery custom Object object error.

Right, let's go back to the Assistant code and get this thing rolling now that we have a basic database available and I don't have to rely on localStorage to keep long-term data.

Nope.

Let's add directories support.

Another Short Detour

So I've started working on the assistant again, but then quickly decided to sidetrack again to adding proper directory support to the file navigator.

Up until now I've mostly used Djot in a testing kind of setup just to make sure basic stuff works, and the most "advanced" djot I've done thus far is the Code Reviewer which is just a few files long.

But coming in to developing this assistant app I already need directories to store the OpenAI API classes and stow them away from view as the just-getting-started 4 files of it or already cluttering the view - I don't want to deal with just a flat list of files trying to build an actual app. So sidetrack it is.

I already had some basic directory support baked in from previous attempts - but I've hidden it behind a toggle flag as it was flaky at best in most situations.

Fast forward four days later, I've implemented directories support, along with improved visuals for the file navigator, copy and paste support (both between djots and copying files from the operating system into a djot).

It was interesting diving into the details of how to keep and retriev a file pointer from the clipboard using the NSFilenamesPboardType clipboard entry, and eventually I've settled on using the electron-clipboard-ex which comes with binaries as writing to this clipboard entry from electron is not supported yet. A lot of useful details in this github issue thread on the Electorn repo.

I'll wrap this entry for now as it is already too long, but you can expect the Firestore database and the new File Navigator improvements in the upcoming Djot release!