Meet DuckGPT: How ChatGPT Became My Rubber Duck (and made me young again)
Rubber duck debugging is an interesting concept that many developers are familiar with. The idea originates from a method in which a developer explains their code or problem to an inanimate object (traditionally a rubber duck) to clarify their own thoughts. The act of explaining forces you to think through the problem logically, often leading to a solution.
On May 18th 2023 OpenAI Introduced the ChatGPT app for IOS, with many features including DuckGPT (it's not really call that - but it integrates a speech-recognition system that enables voice input and it syncs your history across devices),
I have a little dog named "Leo" that I walk many times a day. I put on my Apple AirPods Max, load up some podcasts, and away we go. During these walks, I come up with some of my best ideas. I used to use the voice memo app to save my thoughts, but now I use the ChatGPT app. One day, when we returned home, I forgot to turn off the app before starting a debugging session, working on some complex sql issue, and started talking to my rubber duck. When it answered my questions verbally and displayed that answer on the browser, it was magical, the hatching of DuckGPT (a rubber duck the talks back)
A practical example
In the open-source GridLinks project, the time had come to introduce a data store and expose its content using RBAC (role-based access control) through an auth-protected microservice. While the method of picking the data store is fodder for another article altogether, let's just assume PostgreSQL was a very reasonable choice.
Gridlinks is a chrome extension that defines a named grid of links as a set of 8 panels, each containing a maximum of 8 rows and up to 4 columns per row for a total of 256 links per grid. A user may have access to an unlimited number of grids but should be able to choose from up to N active grids at any given time (for now N will be 4). I envision that a user picks 4 grids and a backend AI process is busy creating more grids based on the a users activity.
We already know that the extension consumes some JSON data which is packaged in panels, and we have the definition of at least fifteen of the panels to date.
The data for a panel is semi-structured and can be described by a pretty simple JSON schema.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name:": {"type": "string" },
"title": {"type": "string" },
"href1": {"type": "string" },
"href2": {"type": "string" },
"links": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"label":{ "type": "string" },
"pre1": {"type": "string" },
"pre2": {"type": "string" },
"pre3": {"type": "string" },
"pre4": {"type": "string" },
"post1": {"type": "string" },
"post2": {"type": "string" },
"post3": {"type": "string" },
"post4": {"type": "string" },
"name1": {"type": "string"},
"href1": {"type": "string"},
"title1": {"type": "string"},
"name2": {"type": "string"},
"href2": {"type": "string"},
"title2": {"type": "string"},
"name3": {"type": "string"},
"href3": {"type": "string"},
"title3": {"type": "string"},
"name4": {"type": "string"},
"href4": {"type": "string"},
"title4": {"type": "string"},
}
}
],
"minItems": 8,
"maxItems": 8
}
},
"required": ["title1", "href1", "href2", "links"]
}
So with this overview in place, we can think about how to design our datastore to efficiently deliver per-user content.
Enter the Duck
Given our definition of the grid from above, create an efficient datastore using PostgreSQL. Examining this conceptual schema reveals a pretty normalized structure; however, you will quickly notice the missing definitions for any data.
In PostgreSQL, jsonb
is a data type that stores JSON (JavaScript Object Notation) data in a binary format, allowing for efficient querying and indexing. Unlike the json
type, jsonb
eliminates whitespace, stores keys in a predefined order, and supports indexing, making it better suited for performance-intensive operations involving JSON data. This type is ideal for scenarios where you need to store and query structured, semi-structured, or unstructured data.
Let's create the database using the conceptual schemas and jsonb
as a means to store our panel data.
CREATE TABLE
users (
id SERIAL PRIMARY KEY,
name VARCHAR(255) UNIQUE NOT NULL,
role VARCHAR(255) DEFAULT 'user',
dbu VARCHAR(255) DEFAULT CURRENT_USER,
created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE
grids (
gid SERIAL PRIMARY KEY,
name VARCHAR(255) UNIQUE NOT NULL,
panels JSONB,
dbu VARCHAR(255) DEFAULT CURRENT_USER,
created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE
usergrids (
ugid SERIAL PRIMARY KEY,
id INT REFERENCES users(id) ON DELETE CASCADE,
gid INT REFERENCES grids(gid) ON DELETE CASCADE,
isactive BOOLEAN DEFAULT TRUE,
dbu VARCHAR(255) DEFAULT CURRENT_USER,
created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE
panels (
pid SERIAL PRIMARY KEY,
name VARCHAR(255) UNIQUE NOT NULL,
links JSONB,
dbu VARCHAR(255) DEFAULT CURRENT_USER,
created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- DATA LOAD
INSERT INTO users (name,role) VALUES
('unknown','user'),('codemarc','admin'),('info','user');
INSERT INTO panels (name, links) VALUES
('amzn', '{"name":"amzn","title":"amazon","href1":"https://aws.amazon.com/","href2":"*/settings","links":[{"pre1":"ㅤ","name1":"alexa","href1":"https://alexa.amazon.com/","post1":"✫","name2":"prime","href2":"https://amazon.com/","post2":"✫","name3":"video","href3":"https://www.amazon.com/Amazon-Video/b/?ie=UTF8&node=2858778011&ref_=nav_dvm_us_eg_cs_nav_rec_T1","post3":"✫","name4":"music","href4":"https://music.amazon.com/"},{"pre1":" ","name1":"audible","href1":"https://www.audible.com/","title1":"Audible","post1":" | ","name2":"twitch","href2":"https://www.twitch.tv/","title2":"Twitch","post2":" | ","name3":"imdb","href3":"https://www.imdb.com/","title3":"IMDb","post3":" | ","name4":"kindle","href4":"https://www.amazon.com/kindle-dbs/fd/kcp","title4":"Kindle Direct Publishing"},{"name1":""},{"name1":"- aws ","href1":"https://aws.amazon.com/"},{"name1":"console","pre1":" ","href1":"https://console.aws.amazon.com/","title1":"AWS Management Console","post1":" ✫ ","name2":"cdk","href2":"https://aws.amazon.com/cdk/","post2":" ✫ ","name3":"js-sdk","href3":"https://aws.amazon.com/sdk-for-javascript/","post3":" ✫ ","name4":"amplify","href4":"https://aws.amazon.com/amplify/"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('apache', '{"name":"apache","title":"apache","href1":"https://apache.org","href2":"*/settings","links":[{"pre1":" projects by » ","name1":"name","href1":"https://projects.apache.org/projects.html?name","post1":" |","name2":"lang","href2":"https://projects.apache.org/projects.html?language","post2":" | ","name3":"cat","href3":"https://projects.apache.org/projects.html?category"},{"name1":"- classics ","title1":"Apache projects that have been around for a long time."},{"pre1":" ","name1":"http","href1":"https://httpd.apache.org/","title1":"The Apache HTTP Server Project is an effort to develop and maintain an open-source HTTP server for modern operating systems including UNIX and Windows NT. The goal of this project is to provide a secure, efficient and extensible server that provides HTTP services in sync with the current HTTP standards.","pre2":" ","name2":"tomcat","href2":"https://tomcat.apache.org/","title2":"The Apache Tomcat software is an open source implementation of the Java Servlet, JavaServer Pages, Java Expression Language and Java WebSocket technologies. The Java Servlet, JavaServer Pages, Java Expression Language and Java WebSocket specifications are developed under the Java Community Process.","pre3":" ","name3":"kafka","href3":"https://kafka.apache.org/","title3":"Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.","pre4":" ","name4":"hadoop","href4":"https://hadoop.apache.org/","title4":"The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage."},{"pre1":" ","name1":"spark","href1":"https://spark.apache.org/","title1":"Apache Spark is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.","pre2":" ","name2":"lucene","href2":"https://lucene.apache.org/","title2":"Apache Lucene is a free and open-source information retrieval software library, originally written completely in Java by Doug Cutting. It is supported by the Apache Software Foundation and is released under the Apache Software License.","pre3":" ","name3":"maven","href3":"https://maven.apache.org/","title3":"Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project''s build, reporting and documentation from a central piece of information.","pre4":" ","name4":"ant","href4":"https://ant.apache.org/","title4":"Apache Ant is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other. The main known usage of Ant is the build of Java applications."},{"name1":"- others","title1":"Apache projects address interesting technology."},{"pre1":" ","name1":"couchdb","href1":"https://couchdb.apache.org/","title1":"Apache CouchDB is a database that uses JSON for documents, an HTTP API, and designed to be accessible via a web browser. It allows for data replication across multiple nodes for robustness and availability, and has a document update model rather than direct overwriting for conflict detection and resolution.","name2":"flume","href2":"https://flume.apache.org/","title2":"Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.","name3":"ambari","hre3":"https://ambari.apache.org/","title3":"Apache Ambari is a software project designed to make Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive collection of operator tools and a robust set of APIs that hide the complexity of Hadoop, simplifying the operation of clusters.","name4":"airflow","href4":"https://airflow.apache.org/","title4":"Apache Airflow is an open-source workflow management platform. It started at Airbnb in October 2014 as a solution to manage the company''s increasingly complex workflows. From monitoring data pipelines to provisioning large-scale data processing resources in a cloud, Airflow is becoming a highly efficient way of writing, scheduling and monitoring workflows."},{"pre1":" ","name1":"parquet","href1":"https://parquet.apache.org/","title1":"Apache Parquet is a columnar storage format that enables efficient storage and retrieval of large analytic datasets. It uses the record shredding and assembly algorithm described in the Dremel paper by Google.","post1":" ","name2":"flink","href2":"https://flink.apache.org/","title2":"Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments, perform computations at in-memory speeds and at any scale.","post2":" ","name3":"thrift","href3":"https://thrift.apache.org/","title3":"Apache Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.","post3":" ","name4":"avro","href4":"https://avro.apache.org/","title4":"Apache Avro is a data serialization system which provides rich data structures, a compact, fast, binary data format, a container file to store persistent data, and remote procedure call (RPC)."},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('banking', '{"name":"banking","title":"banking","href1":"https://www.bankingdive.com/","href2":"*/settings","links":[{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"chase","href1":"https://chaseonline.chase.com/","name2":"bofa","href2":"https://www.bankofamerica.com/","name3":"wells fargo","href3":"https://wellsfargo.com/","name4":"citi","href4":"https://online.citi.com/US/ag/dashboard/credit-card"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"usbank","href1":"https://www.usbank.com/index.html","name2":"pnc","href2":"https://www.pnc.com/en/personal-banking.html","name3":"truist","href3":"https://www.truist.com/","name4":"capone","href4":"https://myaccounts.capitalone.com/welcome"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"fifththird","href1":"https://www.53.com/content/fifth-third/en/login.html","name2":"td","href2":"https://onlinebanking.tdbank.com/","name3":"bmo","href3":"https://www.bmo.com/en-us/main/personal/","name4":"citizens","href4":"https://www.citizensbank.com/"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"1st citzens","name2":"m&t","name3":"huntington nb"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"amex","href1":"https://www.americanexpress.com/en-us/account/login?inav=iNavLnkLog","name2":"key Bank","name3":"ally Bank","name4":"hsbc"},{"name1":"-cards"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"discover","href1":"https://portal.discover.com/customersvcs/universalLogin/ac_main","name2":"apple","href2":"https://card.apple.com","name3":"ignite","href3":"https://cards.theignitecard.com/dfc/dashboard"},{"pre1":"~ syncbank ⇒ ","post1":"ㅤ|ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"amazon","href1":"https://www.synchrony.com/accounts/?client=amazon","name2":"others","href2":"https://www.synchrony.com/accounts/"},{"pre1":" "}]}'),
('cloud', '{"name":"cloud","title":"cloud","href1":"https://www.cloudzero.com/blog/cloud-service-providers/","href2":"*/settings","links":[{"pre1":"ㅤ ","post1":"ㅤㅤ","post2":"ㅤㅤ","post3":"ㅤㅤ","name1":"aws","href1":"https://console.aws.amazon.com/","name2":"azure","href2":"https://portal.azure.com/#home","name3":"gcp","href3":"https://console.cloud.google.com/","name4":"do","href4":"https://cloud.digitalocean.com/login"},{"pre1":"ㅤ ","post1":"ㅤㅤ","post2":"ㅤㅤ","post3":"ㅤㅤ","name1":"-rancher/docs","href1":"https://rancher.com/docs/"},{"pre1":"ㅤ ","post1":"ㅤㅤ","post2":"ㅤㅤ","post3":"ㅤㅤ","name1":"rancher","href1":"http://localhost","name2":"gridsvcs","href2":"https://localhost/dashboard/c/local/explorer/namespace/gridsvcs#Resources","name3":"fusionauth","href3":"http://localhost:9011/"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"name1":"-research"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"fusionauth","href1":"https://fusionauth.io","name2":"passportjs","href2":"https://www.passportjs.org/","name3":"cloudflare","href3":"https://developers.cloudflare.com/products/"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('comms', '{"name":"comms","title":"communicate / discuss","href1":"https://www.youtube.com/@allin","href2":"*/settings","links":[{"pre1":" ","post1":"ㅤ","post2":"|","name1":"microsoft 365","href1":"https://www.microsoft.com/en-us/microsoft-365","name2":"work","href2":"https://www.office.com/?auth=2","name3":"home","href3":"https://www.office.com/?auth=1"},{"pre1":" ","post1":"ㅤ","pre3":"|","name1":"google office","href1":"https://admin.google.com/?authuser=1","name2":"gmail","href2":"https://mail.google.com/mail/u/0/?tab=rm&ogbl#inbox","name3":"dashboard","href3":"https://mail.google.com/mail/u/0/?tab=rm&ogbl#inbox"},{"pre1":"ㅤ","post1":null,"post2":null,"post3":null,"name1":"onedrive","href1":"https://onedrive.live.com","name2":"dropbox","href2":"https://www.dropbox.com/","name3":"g-drive","href3":"https://drive.google.com/drive/u/0/","name4":"icloud","href4":"https://www.icloud.com/iclouddrive/"},{"pre1":"ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"zoom","href1":"https://zoom.us/","title1":"Zoom Video Communications","name2":"slack","href2":"https://slack.com/","title2":"Slack: Where Work Happens","name3":"meet","href3":"https://meet.google.com/","title3":"Google Meet: Secure Video Meetings","name4":"teams","href4":"https://www.microsoft.com/en-us/microsoft-teams/group-chat-software","title4":"Microsoft Teams: Group Chat Software"},{"pre1":"ㅤ"},{"name1":"-articles"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","name1":"reddit","href1":"https://www.reddit.com/","name2":"substack","href2":"https://substack.com/home","name3":"medium","href3":"https://medium.com/","name4":"notion","href4":"https://www.notion.so","title4":"Notion.so is a popular productivity tool that can be described as an all-in-one workspace. It lets you take notes, create documents, manage tasks, and even build wikis and databases – all in a single platform."},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"dev.to","href1":"https://dev.to/","name2":"quora","href2":"https://www.quora.com/topic/Software-Engineering","name3":"groups","href3":"https://www.linkedin.com/groups/","name4":"hacker news","href4":"https://news.ycombinator.com"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('devops', '{"name":"devops","title":"devops toolchain","href1":"https://www.cncf.io/","href2":"*/settings","links":[{"pre1":" ","name1":"docker","href1":"https://www.docker.com/","name2":"kubernetes","href2":"https://kubernetes.io/","name3":"k3s","href3":"https://k3s.io/","name4":"helm","href4":"https://helm.sh/"},{"pre1":" ","name1":"datadog","href1":"https://www.datadoghq.com/","name2":"prometheus","href2":"https://prometheus.io/","name3":"grafana","href3":"https://grafana.com/","name4":"elk","href4":"https://www.elastic.co/elastic-stack"},{"name1":"pulumi","href1":"https://www.pulumi.com/","name2":"serverless","href2":"https://www.serverless.com/","name3":"terraform","href3":"https://www.terraform.io/","name4":"ansible","href5":"https://www.ansible.com/"},{"pre1":"ㅤㅤ","name1":"istio","href1":"https://istio.io/","name2":"linkerd","href2":"https://linkerd.io/","name3":"consul","href3":"https://www.consul.io/","name4":"envoy","href4":"https://www.envoyproxy.io/"},{"pre1":"ㅤ","name1":"terraform","href1":"https://www.terraform.io/","name2":"ansible","href2":"https://www.ansible.com/","name3":"puppet","href3":"https://puppet.com/","name4":"chef","href4":"https://www.chef.io/"},{"pre1":"ㅤ","name1":"jenkins","href1":"https://www.jenkins.io/","name2":"gitLab ci","href2":"https://docs.gitlab.com/ee/ci/","name3":"circle ci","href3":"https://circleci.com/","name4":"travis ci","href4":"https://travis-ci.org/"},{"pre1":"ㅤ","name1":"rancher","href1":"https://rancher.com/","name2":"openshift","href2":"https://www.openshift.com/","title2":"Red Hat OpenShift is a container application platform that allows developers and IT organizations to build, deploy, and manage applications consistently across on-premises, public, and private cloud infrastructures. It is based on open-source Kubernetes container orchestration technology and provides a comprehensive set of tools and services for developing, deploying, and managing containerized applications.","name3":"nomad","href3":"https://www.nomadproject.io/","title3":"A simple and flexible scheduler and orchestrator to deploy and manage containers and non-containerized applications across on-premises and clouds at scale."},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('devtools', '{"name":"devtools","title":"devtools","href1":"https://stackshare.io/tools","href2":"*/settings","links":[{"pre1":" ","post1":" | ","post2":" | ","post3":" | ","name1":"book","href1":"https://book.git-scm.com/book/en/v2","name2":"cheats","href2":"https://gist.github.com/codemarc/7e3c37a880e679f8fa310f67cdb29a85","name3":"gists","href3":"https://gist.github.com/","name4":"insider","href4":"hrefhttps://resources.github.com/github-insider/practitioner/4"},{"pre1":" ","post1":" | ","post2":" | ","post3":" | ","name1":"tables","href1":"https://tableconvert.com/","name2":"mui","href2":"https://mui.com/material-ui/all-components/","name3":"jwt","href3":"https://jwt.io/#algorithm-select","name4":"unicode","href4":"https://compart.com/en/unicode"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"roadmap","href1":"https://roadmap.sh/","name2":"bash","href2":"https://devhints.io/bash","name3":"css","href3":"https://htmlcheatsheet.com/css/","name4":"rpt","href4":"https://codemarc.net/report.html"},{"label":"-"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"postman","href1":"https://www.postman.com/","name2":"sonarqube","href2":"https://www.sonarqube.org/","name3":"goaccess","href3":"https://goaccess.io/"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"rabbitmq","href1":"https://www.rabbitmq.com/","name2":"zeromq","href2":"https://zeromq.org/","name3":"suprabase","href3":"https://supabase.com/dashboard/projects"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('fintech', '{"name":"fintech","title":"financial news / data","href2":"*/settings","links":[{"pre1":"ㅤ","name1":"yahoo","href1":"https://finance.yahoo.com","title1":"Market data, news, and more.","pre2":"ㅤ","name2":"google","href2":"https://www.google.com/finance","title2":"Aggregates financial news and market data from multiple sources.","pre3":" ","name3":"seeking alpha","href3":"https://seekingalpha.com","title3":"Crowd-sourced financial content service. In-depth analysis and market commentary."},{"pre1":"ㅤ","name1":"bloomberg","href1":"https://www.bloomberg.com","title1":"Extensive financial news, data, and analysis.Comprehensive financial coverage.","pre2":"ㅤ","name2":"marketwatch","href2":"https://www.marketwatch.com","title2":"Breaking news and stock market updates.","pre3":" ","name3":"wsj","href3":"https://www.wsj.com","title3":"Comprehensive financial news."},{"pre1":"ㅤ","name1":"reuters","href1":"https://www.reuters.com/finance","title1":"Global financial coverage.","name2":"investing.com","href2":"https://www.investing.com","title2":"Real-time financial data and tools.","name3":"nasdaq","href3":"https://nasdaq.com","title3":"nasdaq market site"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"name1":"-finviz","href1":"https://finviz.com/"},{"pre1":" ","post1":" ","post2":"ㅤ","post3":"ㅤ","name1":"schwab","href1":"https://client.schwab.com/app/accounts/positions/#/","name2":"fidelity","href2":"https://digital.fidelity.com/prgw/digital/login/full-page","name3":"robinhood","href3":"https://robinhood.com/login/"},{"pre1":" ","post1":" ","post2":"ㅤ","post3":"ㅤ","name1":"yieldmax","href1":"https://www.yieldmaxetfs.com/","name2":"cnbc","href2":"https://www.cnbc.com/live-tv/?trknav=navigation:shows:watchlive:103620081","name3":"cnn mkts","href3":"https://www.cnn.com/markets"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('genai', '{"name":"genai","title":"genai / tools","href1":"https://venturebeat.com/category/ai/","href2":"*/settings","links":[{"pre1":" ","name1":"chatgpt","href1":"https://chat.openai.com/","title1":"OpenAI''s ChatGPT","name2":"claude","href2":"https://claude.ai/chats","title2":"Anthropic''s Claude","name3":"gemini","href3":"https://gemini.google.com/","title3":"Google''s Gemini","name4":"perplexity","href4":"https://www.perplexity.ai/","title4":"Perplexity AI Search"},{"pre1":" ","name1":"meta.ai","href1":"https://www.meta.ai/","title1":"Meta''s AI Research","name3":"x.ai","href3":"https://x.ai/","title3":"x.ai","name2":"hugging face","href2":"https://huggingface.co/","title2":"Hugging Face AI Community"},{"name1":"-art/image"},{"pre1":" ","name1":"dall-e","href1":"https://openai.com/dall-e-2","title1":"OpenAI''s DALL-E, generates highly realistic and creative images from text descriptions. Known for its versatility and ability to understand complex prompts. Offers editing features like inpainting and outpainting.","name2":"midjourney","href2":"https://www.midjourney.com/","title2":"Midjourney AI Art. Produces high-quality, artistic images from text prompts. Known for its distinctive aesthetic, often described as painterly or illustrative. Operates primarily through a Discord bot interface. Offers various style options and aspect ratios","name3":"stable diffusion","href3":"https://stability.ai/","title3":"Stability AI''s Stable Diffusion, Open-source image generation model. Can run locally on consumer hardware, unlike many cloud-based alternatives. Highly customizable with a large community creating custom models and implementations. Capable of various tasks including text-to-image, image-to-image, and inpainting"},{"pre1":" ","name1":"runwayml","href1":"https://runwayml.com/","title1":"Runway AI Video Editor - Offers AI-powered video editing and generation. Known for text-to-video capabilities and advanced editing features","name2":"digital-id","href2":"https://www.d-id.com/creative-reality-studio/","title2":"Specializes in creating talking head videos from still images. Offers realistic lip-sync and facial animations","name3":"designer","href3":"https://designer.microsoft.com/","title3":"Microsoft"},{"name1":"- coding "},{"pre1":" ","name1":"copilot","href1":"https://github.com/features/copilot","title1":"GitHub Copilot","name2":"codewhisperer","href2":"https://aws.amazon.com/codeguru/code-whisperer/","title2":"Amazon CodeWhisperer - integrated with AWS services","name3":"sourcegraph","href3":"https://sourcegraph.com/search"},{"pre1":"~ lamatic ","post1":" ","post2":" ","post3":" ","name1":"docs","href1":"https://lamatic.ai/docs","name2":"studio","href2":"https://studio.lamatic.ai/"},{"pre1":"ㅤ"}]}'),
('goog', '{"name":"goog","title":"google","href1":"https://developers.google.com/","href2":"*/settings","links":[{"pre1":"ㅤ","name1":"cloud console","href1":"https://console.cloud.google.com/","title1":"Google Cloud Console","name2":"compute engine","href2":"https://cloud.google.com/compute","title2":"Google Compute Engine","name3":"app engine","href3":"https://cloud.google.com/appengine","title3":"Google App Engine","name4":"cloud functions","href4":"https://cloud.google.com/functions","title4":"Google Cloud Functions"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('house', '{"name":"house","title":"household / credit","href1":"https://home.nest.com/home","href2":"*/settings","links":[{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"flagstar","href1":"https://www.flagstar.com/myloans/login","title1":"mortgage","name2":"campbell","href2":"https://portal.campbellproperty.com/dashboard","title2":"hoa property manager","name3":"mizner cc","href3":"https://www.miznercountryclub.org/","title3":"mizner"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"fpl","href1":"https://www.fpl.com/","title1":"florida power and light","name2":"water","href2":"https://ebill.pbcwater.com/Security/loginui.aspx","title2":"palm beach county water","name3":"xfinity","href3":"https://customer.xfinity.com/","name4":"verizon","href4":"http://my.verizon.com/"},{"pre1":" "},{"pre1":" "},{"pre1":" "},{"name1":"-education","href1":"https://studentaid.gov/h/apply-for-aid/fafsa"},{"pre1":" ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"one.uf","href1":"https://one.uf.edu/","name2":"dpe","href2":"https://www.billhighway.com/aph/forChapters/v2/login.aspx","name3":"gru","href3":"https://myaccount.gru.com","name4":"trimark","href4":"https://trimarkp.twa.rentmanager.com/Shared/Login"},{"pre1":" ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"nelnet","href1":"https://nelnet.studentaid.gov/welcome","name2":"aidvantage","href2":"https://aidvantage.studentaid.gov/","name3":"salliemae","href3":"https://www.salliemae.com/login/"},{"pre1":" "}]}'),
('msft', '{"name":"msft","title":"microsoft","href1":"https://developer.microsoft.com/","href2":"*/settings","links":[{"pre1":" ","name1":"learn","href1":"https://docs.microsoft.com/learn/","title1":"Microsoft Learn","post1":" | ","name2":"docs","href2":"https://docs.microsoft.com/","title2":"Microsoft Docs","post2":" | ","name3":"q&a","href3":"https://docs.microsoft.com/answers/","title3":"Microsoft Q&A","post3":" | ","name4":"dev blog","href4":"https://devblogs.microsoft.com/","title4":"Microsoft Developer Blogs"},{"pre1":" ㅤ","name1":"vs code","href1":"https://code.visualstudio.com/docs","title1":"Visual Studio Code","name2":".net","href2":"https://dotnet.microsoft.com/","title2":".NET","name3":"nuget","href3":"https://www.nuget.org/","title3":"NuGet","name4":"mssql","href4":"https://www.microsoft.com/sql-server/","title4":"SQL Server"},{"name1":""},{"name1":"- azure ","href1":"https://azure.microsoft.com/"},{"name1":"devops","href1":"https://azure.microsoft.com/services/devops/","title1":"Azure DevOps","name2":"functions","href2":"https://azure.microsoft.com/services/functions/","title2":"Azure Functions","name3":"ak8ss","href3":"https://azure.microsoft.com/services/kubernetes-service/","title3":"Azure Kubernetes Service (AKS)","name4":"ml studio","href4":"https://studio.azureml.net/","title4":"Azure Machine Learning Studio"},{"pre1":" ","name1":"identity platform","href1":"https://azure.microsoft.com/services/active-directory/","title1":"Microsoft Identity Platform","pre2":"ㅤㅤㅤㅤ","name2":"graph","href2":"https://developer.microsoft.com/graph/","title2":"Microsoft Graph"},{"label":"-"},{"pre1":"~ power ","name1":"shell","href1":"https://docs.microsoft.com/powershell/","title1":"PowerShell","post1":" ","name2":"apps","href2":"https://powerapps.microsoft.com/","title2":"Power Apps","post2":" ","name3":"automate","href3":"https://flow.microsoft.com/","title3":"Power Automate","post3":" ","name4":"bi","href4":"https://powerbi.microsoft.com/","title4":"Power BI"},{"pre1":"ㅤ"},{"name1":" "}]}'),
('services', '{"name":"services","title":"services","href1":"https://codemarc.net/doc/gridsvcs/#/","href2":"*/settings","links":[{"pre1":" gridsvcs: ","post1":" ","post2":" ","name1":"dev","href1":"http://localhost:3000/v1/api-docs/","name2":"test","href2":"https://codemarc.net/api/v1/api-docs/","name3":"prod","href3":"http://localhost:3000/v1/api-docs/"},{"pre1":""},{"pre1":"~ deployments"},{"pre1":" dev: ","post1":" |","post2":" |","name1":"status","href1":"http://localhost:3000/v1/motd/status","name2":"quotes","href2":"http://localhost:3000/v1/motd/quotes","name3":"topics","href3":"http://localhost:3000/v1/motd/topics"},{"pre1":" test: ","post1":" |","post2":" |","name1":"status","href1":"https://codemarc.net/api/v1/motd/status","name2":"quotes","href2":"https://codemarc.net/api/v1/motd/quotes","name3":"topics","href3":"http://codemarc.net/api/v1/motd/topics"},{"pre1":" prod: ","post1":" |","post2":" |","name1":"status","href1":"http://localhost:3000/v1/motd/status","name2":"quotes","href2":"http://localhost:3000/v1/motd/quotes","name3":"topics","href3":"http://localhost:3000/v1/motd/topics"},{"pre1":""},{"pre1":"~ projects"},{"pre1":"ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"gridlinks","href1":"https://github.com/users/codemarc/projects/6","name2":"gridsvcs","href2":"https://github.com/users/codemarc/projects/5"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('shopify', '{"name":"shopify","title":"shopify","href1":"https://www.shopify.com/","title2":"status","href2":"https://www.shopifystatus.com/","links":[{"pre1":" ","name1":"the puppy leo shop","href1":"http://puppyleo.com","pre2":"ㅤ🐶ㅤ","name2":"admin","href2":"https://admin.shopify.com/"},{"pre1":"ㅤ","name1":"zendrop","href1":"https://app.zendrop.com/dashboard","pre2":"ㅤ","name2":"cj","href2":"https://www.cjdropshipping.com/","pre3":"ㅤ","name3":"shineon","href3":"https://www.shineon.com/\"","pre4":"ㅤ","name4":"printful","href4":"https://www.printful.com/"},{"pre1":"ㅤㅤ","name1":"summer 2024","href1":"https://www.shopify.com/editions/summer2024","pre2":"ㅤ","name2":"changelog","href2":"https://shopify.dev/changelog"},{"name1":" src ","href1":"https://shopify.github.io","post1":" ◦ ","name2":" dev ","href2":"https://shopify.dev/","post2":" ◦ ","name3":"apps","href3":"https://apps.shopify.com/","title3":"Shopify App Store","post3":" ◦ ","name4":"community","href4":"https://community.shopify.com/c/shopify-community/ct-p/en","title4":"Shopify Community Forums"},{"pre1":"ㅤ","name1":"api","href1":"https://shopify.dev/api","title1":"Shopify API Reference","pre2":" ","name2":"polaris","href2":"https://polaris.shopify.com/","title2":"Shopify Polaris - React Components","pre3":" ","name3":"partners","href3":"https://help.shopify.com/en/partners","pre4":" ","name4":"academy","href4":"https://www.shopify.com/partners/academy","title4":"Shopify Partner Academy"},{"pre1":" ","name1":"themekit","href1":"https://shopify.github.io/themekit/","title1":"Shopify Theme Kit","post1":"|","name2":"bridge","href2":"https://shopify.dev/tools/app-bridge","title2":"Shopify App Bridge","post2":"|","name3":"authflow","href3":"https://shopify.dev/apps/auth/oauth","title3":"Implementing OAuth for Shopify Apps","post3":"|","name4":"social","href4":"https://buffer.com/library/social-media-sites/#20-tumblr-%E2%80%94-135-million-maus","title4":"buffer.com Buffer helps you build an audience organically. We’re a values-driven company that provides affordable, intuitive marketing tools for ambitious people and teams."},{"name1":"arch","href1":"https://shopify.dev/themes/architecture","title1":"Shopify Theme Architecture","post1":"|","name2":"templates","href2":"https://shopify.dev/themes/architecture/templates","title2":"Shopify Theme Templates","post2":"|","name3":"sections","href3":"https://shopify.dev/themes/architecture/sections","title3":"Shopify Theme Sections","post3":"|","name4":"inspector","href4":"https://shopify.dev/themes/tools/theme-inspector","title4":"Shopify Theme Inspector for Chrome"},{"pre1":"ㅤㅤ","name1":"nodesdk","href":"https://github.com/Shopify/shopify-node-api","post1":"✫","name2":"store","href2":"https://themes.shopify.com/","post2":"✫","name3":"exchange","href3":"https://exchange.shopify.com/","title3":"Shopify Exchange Marketplace"},{"name1":"ㅤ"}]}'),
('stream', '{"name":"stream","title":"stream / shop","href1":"https://www.xfinity.com/stream/","href2":"*/settings","links":[{"pre1":" ","post1":"✫","post2":"✫","post3":"✫","name1":"alexa","href1":"https://alexa.amazon.com/","name2":"prime","href2":"https://amazon.com/","name3":"video","href3":"https://www.amazon.com/Amazon-Video/b/?ie=UTF8&node=2858778011&ref_=nav_dvm_us_eg_cs_nav_rec_T1","name4":"music","href4":"https://music.amazon.com/"},{"pre1":" ","name1":"audible","href1":"https://www.audible.com/","title1":"Audible","post1":" | ","name2":"twitch","href2":"https://www.twitch.tv/","title2":"Twitch","post2":" | ","name3":"imdb","href3":"https://www.imdb.com/","title3":"IMDb","post3":" | ","name4":"kindle","href4":"https://www.amazon.com/kindle-dbs/fd/kcp","title4":"Kindle Direct Publishing"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"xfinity","href1":"https://www.xfinity.com/stream/","name2":"netflix","href2":"https://www.netflix.com/","name3":"hbomax","href3":"https://hbomax.com/","name4":"peacock","href4":"http://peacock.tv/"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"youtube","href1":"https://www.youtube.com/","name2":"apple music","href2":"https://music.apple.com/","name3":"spotify","href3":"https://www.spotify.com/"},{"pre1":" ","name1":"-shopify","href1":"https://www.shopify.com/editions/summer2024","xhref1":"https://www.shopify.com/"},{"pre1":" ","post2":" ","post3":" ","name1":"the puppy leo shop","href1":"http://puppyleo.com","name2":" 🐶 admin","href2":"https://admin.shopify.com/"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"zendrop","href1":"https://app.zendrop.com/dashboard","name2":"cj","href2":"https://www.cjdropshipping.com/","name3":"shineon","href3":"https://www.shineon.com/\"","name4":"printful","href4":"https://www.printful.com/"},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"src","href1":"https://shopify.github.io","name2":"dev","href2":"https://shopify.dev/","name3":"apps","href3":"https://apps.shopify.com/","title3":"Shopify App Store","name4":"polaris","href4":"https://polaris.shopify.com/","title4":"Shopify Polaris - React Components"},{"pre1":" ","post1":" ","post2":" ","post3":" "},{"pre1":" ","post1":" ","post2":" ","post3":" "}]}'),
('blank', '{"name":"blank","title":"panel","href1":"https://codemarc.net/doc/gridlinks/#/","href2":"*/settings","links":[{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r1c1","name2":"r1c2","name3":"r1c3","name4":"r1c4"},{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r2c1","name2":"r2c2","name3":"r2c3","name4":"r2c4"},{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r3c1","name2":"r3c2","name3":"r3c3","name4":"r3c4"},{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r4c1","name2":"r4c2","name3":"r4c3","name4":"r4c4"},{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r5c1","name2":"r5c2","name3":"r5c3","name4":"r5c4"},{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r6c1","name2":"r6c2","name3":"r6c3","name4":"r6c4"},{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r7c1","name2":"r7c2","name3":"r7c3","name4":"r7c4"},{"pre1":" ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r8c1","name2":"r8c2","name3":"r8c3","name4":"r8c4"},{"pre1":"ㅤ"}]}')
;
INSERT INTO
grids (name, panels)
VALUES
('builtin', '[1,2,3,4,5,6,7,8]'),
('personal', '[1,9,10,11,3,12,13,14]');
INSERT INTO
usergrids (id, gid, isactive)
VALUES
(1, 1, FALSE),
(2, 1, FALSE),
(2, 2, TRUE),
(3, 1, FALSE);
The Power of Asking the Right Questions
How do we convert our hand coded yaml files into database tables? Ask the duck (ala copilot):
Can we create a node based script named y2j that runs js-yaml.js on files in the data/yaml folder and pipes stdout to the data/json folder?
Given the table "panels", I want to create a script named j2s that takes json files prepared by y2j and converts them into a sql insert statement?
These prompts let the duck spit out complete scripts that with a few minor tweaks that accomplished my goals in less time then it has taken to explain it.
Now thinking about how to get data in make you think about how to get data out.
The Challenge of JSONB Queries
JSONB, with its ability to store JSON data in a binary format, offers a powerful tool for developers. It allows for efficient storage, indexing, and querying of JSON data in PostgreSQL. However, writing queries that leverage this power can be tricky. The syntax can be complex, and ensuring that you're accessing and manipulating the data correctly often requires a deep understanding of both SQL and the structure of your JSON data
You can use the arrow operators in a sql query:->
Use this when you need to access nested JSON structures or when you want to work with the data as JSON (e.g., further JSON manipulation)
->>
Use this when you want to extract the value as plain text (e.g., for comparison in SQL queries or when you don't need to maintain JSON formatting).
SELECT data->'key' AS value FROM your_table
WHERE data->>'another_key' = 'some_value';
When faced with a challenging SQL query, especially one involving JSONB, I often turn to DuckGPT. But it’s not just about getting the right answer—it's about knowing how to ask the right question. This is where the process becomes both an art and a science.
Query 1
SELECT
u.name AS username,
COALESCE(json_agg(g.name), '[]'::json ) AS gridset
FROM
cmc.users u
JOIN
cmc.usergrids ug ON u.id = ug.id
JOIN
cmc.grids g ON ug.gid = g.gid
GROUP BY
u.id, u.name;
Explanation: This query selects the username and a JSON array (
gridset
) of grid names associated with each user. If a user has no grids, theCOALESCE
function ensures that an empty JSON array ('[]'::json
) is returned.Use Case: This is a general query that can be used to retrieve all users and their associated grids. It's flexible and doesn't filter out inactive grids, making it useful when you want a comprehensive overview of user-grid associations.
Query 2
SELECT
u.name AS username,
COALESCE(json_agg(g.name) FILTER (WHERE ug.isactive = TRUE), '[]'::json ) AS gridset
FROM
cmc.users u
JOIN
cmc.usergrids ug ON u.id = ug.id
JOIN
cmc.grids g ON ug.gid = g.gid
GROUP BY
u.id, u.name;
Explanation: This query is similar to the first but adds a
FILTER
clause to include only active grid associations (ug.isactive = TRUE
). It also ensures that users without active grids receive an empty JSON array.Use Case: This query is more refined, focusing on active associations only. It’s particularly useful for scenarios where only current, active grid associations are relevant. This aligns well with Gridlinks' objective to display up-to-date, relevant information for users.
Query 3
SELECT
u.name AS username,
COALESCE(json_agg(g.name) FILTER (WHERE ug.isactive = TRUE), '[]'::json ) AS gridset
FROM
cmc.users u
JOIN
cmc.usergrids ug ON u.id = ug.id
JOIN
cmc.grids g ON ug.gid = g.gid
WHERE
u.name = 'codemarc'
GROUP BY
u.id, u.name;
Explanation: This query is identical to the second one but adds a
WHERE
clause to filter the results for a specific user (u.name
= 'codemarc'
).Use Case: This is a targeted query for when you need to retrieve grid information for a specific user. In the context of Gridlinks, it’s useful for generating user-specific data, such as when a user logs in and the system needs to fetch their grids and associated panels.
And Finally....
SELECT
u.name AS username,
u.role AS userrole,
g.name AS grid_name,
json_agg(
json_build_object(
'panel_name', p.name,
'links', p.links
)
ORDER BY array_position(array(SELECT jsonb_array_elements_text(g.panels)), p.pid::text)
) AS panels
FROM
cmc.users u
JOIN
cmc.usergrids ug ON u.id = ug.id
JOIN
cmc.grids g ON ug.gid = g.gid
JOIN
cmc.panels p ON p.pid = ANY(ARRAY(SELECT jsonb_array_elements_text(g.panels) :: int))
WHERE
u.name = 'codemarc'
AND ug.isactive = TRUE
GROUP BY
u.name, u.role, g.name, g.panels;
Explanation:
u.name
AS username
: Retrieves the username.u.role AS userrole
: Retrieves the user role.g.name
AS grid_name
: Retrieves the name of each grid.json_agg(...) AS panels
: Aggregates the panels into a JSON array of objects, each containing the panel name and its links.ORDER BY array_position(...)
: Orders the panels in the JSON array based on the position specified in thepanels
field of thegrids
table.JOIN
clauses: Connects theusers
,usergrids
,grids
, andpanels
tables.p.pid
= ANY(...)
: Filters the panels based on the list of panel IDs in thepanels
JSONB array from thegrids
table.WHERE
clause: Filters the results to include only the specified username and active grids.GROUP BY
clause: Groups the results by the username, user role, grid name, and panels field to ensure proper aggregation.
Key Points:
Replace
'codemarc'
with the actual username you're querying for or use a parameter if this query is part of an application.Ordering: The
ORDER BY array_position(...)
ensures that the panels are ordered according to their position in thepanels
JSONB field of thegrids
table.
This query provides a detailed, ordered JSON output that aligns well with the requirements of Gridlinks or any similar application. If there are any further adjustments or clarifications needed, feel free to ask!
Conclusion
DuckGPT has transformed my debugging process, turning what was once a solitary task into an interactive and efficient experience. By leveraging the power of DuckGPT, I can now articulate my problems and receive insightful feedback in real-time, making complex issues more manageable. This innovative approach not only enhances productivity but also brings a sense of youthful curiosity and excitement back into my work. Whether it's converting YAML files to JSON, crafting intricate SQL queries, or simply brainstorming ideas, DuckGPT proves to be an invaluable companion, demonstrating the profound impact of AI on modern development practices.
#RubberDuckDebugging #ChatGPT #DuckGPT #OpenAI #TechInnovation #DeveloperTools #PostgreSQL #TechProductivity #DebuggingTips #AIInDevelopment #TechJourney #ModernDevelopment #VoiceInput #TechEfficiency #AIIntegration #TechExploration #CodingLife
Subscribe to my newsletter
Read articles from Marc Greenberg directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by