Top functional programming based OpenSource technologies
Roman MPurpose
The goals of this article is to research the most popular FP based technologies to prove FP soundness and versatility and to demonstrate that FP has been already with us for a long time. Methodology involved analyzing GitHub for the most popular projects, blog posts and academic papers.
A little bit of historical context
Engaging talk with a lot of references to historical moments emphasizing point that:
"Good" languages are differed from "bad" languages by theoretical base (specifically math logic brought to computer science). Not just engineered.
"Propositions as Types" by Philip Wadler - YouTube
LISP
Ancestor of almost all functional languages. Originally was designed to solve 1 specific problem of symbolic differentiation, for which FORTRAN was very inconvenient. And eventually became general purpose-purpose language. In that sense (growing from 1 specif problem) it is similar to Erlang. With time passed became purer language thanks to academic influence. It had inspired a lot of other languages including the whole family of so-called LISP languages. The second-oldest high-level programming language still in common use, after FORTRAN. It brought to the table such essential concepts without which we can not imagine our industry today as : - Conditionals not limited to goto (if-then-else) - First-class functions - Recursion - Garbage collection - Dynamic typing - Interactive programming environment (REPL) - Full language available at load time, compile time, and run time
Haskell
Appeared in 1990 with stable release at 2010. If LISP started all funtional obsession in XX century then Haskell summarized ideas of languages appeared in the next 30 years. The main idea was not that academics should not reinvent same features in their languages, but to make common ground for benefits of all. Fun part is that writing I/O related code in Haskell was so annoying, so Monadic I/O was ultimately invented that has groundbraking influence. Here stonk monad plot showing Haskell m-wording the world.
Other
There are plenty languages worth to mention and their role in history, but let me focus on some of them relevant today.
OCaml
Direct descendant of ML which Appeared as dissatisfaction with LISP writing proofers. The idea was to have types which lower logical errors. (LISP -> ML -> OCaml) OCaml was written in 1996 in France. It was inspired by a long line of research into ML, starting in the 1960s, and continues to have deep links to the academic community. Less pure comparing to Haskell but more focused on Industry. So if you want to participate in making tooling for whole ecosystem for future generations - it is a good opportunity to learn and make huge impact.
Erlang
Appered from need for faster develompent to reliable run on telephony switches, because in 80s managers realized that they spend more on developing and maintaning software than on hardware itself. It was designed to be concurent, distributed and high available from beginning long time ago before async fashion in JS and Python. In contrast, OTP and Erlang almost 30 years(open-sourced) old battle tested technology. Interesing, that originally it was propriate closed language of Ericsson's company, which is powering world networking now.
Top projects grouped by language
Elixir
Appeared at 2012. Built on top of BEAM to make Erlang\OTP runtime more accessible, because Erlang is not user friendly really; and shares the same abstractions and OTP framework for building distributed, fault-tolerant applications. Though initially designed with dynamic typing, Elixir is actively developing static typing capabilities cnrs. As Joe Armstrong said about types "no good type system save you from node failure" — that is why Erlang language was dynamically typed — it was not focus. And it is one of the two languages in this list available on Leetcode, but I would recommend to use https://exercism.org/ for practice firstly. You can think about this language as more modern and easy to use language for BEAM runtime than original Erlang language. Why does it matter? Because Erlang really powers high load. You can find out more here about real world use cases of Erlang its VM called BEAM. Whats-app cloud handles 2 millions connection per a node for instance. It is simple language nevertheless a powerful one. Maybe Elixir has not really good theoretical fundament, but it is still good language and its author José Valim actively works on bringing type system with synergy of academic world.
Picture from talk by creator of the lang José about different levels of abstractions Keynote: Gang of None? Design Patterns in Elixir - José Valim | ElixirConf EU 2024 - YouTube
Phoenix: Top from stack overflow surveys
Stars: 21k One of the most if not the most popular FP frameworks that has won "the most admired web framework" at stackoverflow research for several years in a row. Stack Overflow Developer Survey 2023 Fullstack, Ruby inspired. Connects high load with easy of use. There are big companies using this framework check related block at Phoenix Framework
analytics
Stars: 19.1k Simple, open source, lightweight (< 1 KB) and privacy-friendly web analytics alternative to Google Analytics. Serious niche and good moral idea. All we can do - is to embrace such kind of projects.
We are dedicated to making web analytics more privacy-friendly.
supabase/realtime
Stars: 6.6k One of the projects of famous supabase project which leverages Phoenix from the first place. It makes use from BEAM through Elixir.
This is a server built with Elixir using the that enables the following functionality:
Broadcast: Send ephemeral messages from client to clients with low latency.
Presence: Track and synchronize shared state between clients.
Postgres Changes: Listen to Postgres database changes and send them to authorized clients.
Erlang/OTP
Not just language but framework too running on BEAM. Erlang is not user friendly or looks familiar to popular languages so Elixir was created to make this incredebly concurrent platform accesable to regular web developers. . Joe Armstrong remarked in an interview with Rackspace in 2013:
"If Java is 'write once, run anywhere', then Erlang is 'write once, run forever'."
CouchDB
Stars: 6.2k
If there’s one word to describe CouchDB, it is relax
Document based reliable scalable database with nicely designed HTTP/JSON interface. With accompanient of Pouchdb can be the best choice for offline-first applications with low effort data syncronisation.
RabbitMQ
Stars: 12.2k
RabbitMQ is a reliable and mature messaging and streaming broker, which is easy to deploy on cloud environments, on-premises, and on your local machine. It is currently used by millions worldwide.
Famious publish/consumer message queue with a lot of scenarios to implement.
emqx
Stars: 14k
The most scalable open-source MQTT broker for IoT, IIoT, and connected vehicles
Gleam
Appeared at 2016. V1 was released at 4 Mar 2024, so it does not have popular projects yet, but it grows fast. Running on Erlang VM as Elixir, but statically typed in opposite of dynamically typed Elixir and Erlang. Inspired by many languages and as author says "I stole the best ideas for this language." © Louis Pilfold . And I have to admit that coding in Gleam is pleasant experience. It is really easy to learn and fast LS gives instant feedback on type coherence. If you can choose among Erlang, Elixir and Gleam. You should deferentially look at Gleam, but if you like Ruby look at Elixir of course. Maybe it is true opensource sponsored language, in this meaning it is a good example of growing contrasting to Elm. Try it at The Gleam Language Tour
P.S. Remember Golang mascot? yeah... Now look at this piece of art.
Front end in Gleam
Lustre allows to write fullstack applications in Gleam reusing same types! Basically it follows the same MVC architecture as Elm.
Clojure
Clojure is a compiled language, yet remains completely dynamic. Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection... is a dialect of Lisp, and shares with Lisp the code-as-data philosophy and a powerful macro system.
Appeared at 2007. From quote we can conclude that one of the main factor becoming popular is running on JVM. Concurent of Scala. Used at Walmart Labs “Our Clojure system just handled its first Walmart black Friday and came out without a scratch.” For me dynamic nature is red flag nowdays, especially when there such beatiful languages such Elm, Ocaml, Haskell and maybe Gleam if it woud not have so many braces. Here are really titans of Clojure world in terms of numbers of stars.
metabase
Stars: 37.4k Very popular tool for work with data for non-technical people. It let to get and visualize simple data without SQL, but for more complex queries SQL is required. One technical person can make views and all non-technical part of a team can use it.
logseq
Stars: 31.1k A privacy-first, open-source platform for knowledge management and collaboration. ==Notion== and Obsidian like alternative.
penpot
Stars: 29.5k The open-source tool for design and code collaboration. ==Figma== alternative. By number of stars we can assume that IT world at least admired this tool.
Elm
Appeared at 2012. Extremely efficient and deligtful to use to build browser UI. Not popular now - in its afterhype period of time, but definitaly impactful. Can be seen as more simple and compact child of Haskell and OCaml. Has elegant type system. Can be learn in a weekend. There are elm-ui, elm-review popular libraries. And there is not much activity on official site actually, but languages is alive. In talk Evan told that he made compiler from Elm to SQL which sounds cool, but he really struggles to make sustainable financial model. Also seems like Evan does not like online marketing his language still loving conferences. Personnaly I agree with him and I think much people prefer real events to virtual ones but it is important lesson for language designers: online presense matters. Check Elmcraft and oficial forum for latest news and inspiration.
Haskell
The most academic language in the list. From the beginning made with good type design comparing to the most other popular languages where types were introduced on the go. Highly admired by creator of ROC language - Richard Feldman. He also praises Elm by the way. This language has a lot of concepts, so it is a good idea to learn language to explore FP ideas in practice. And I would say that language is excellent for well understood problem that can be something formally described to leverage its strong type system. Some researches say that there are better language than Haskell like Agda or Idris. Ok, but for many of FP languages Haskell is a progenitor. The most elegant and concise language in the list, nevertheless it can be unpleasant to use in production due to various ways to do the same thing.
ShellCheck
Stars: 35.6k A static analysis tool for shell scripts adopted by: Travis CI, Codacy, Code Climate, Code Factor, CircleCI via the ShellCheck Orb GitHub (only Linux), Trunk Check (universal linter; allows you to explicitly version your shellcheck install) via the shellcheck plugin And you can use it for too if you work with bash/sh scripts.
pandoc
Stars: 33.3k Universal markup converter. Supports numerous format from/to. Practical and helpful.
PostgREST
Stars: 22.7k
REST API with automatically generated OpenAPI for any PostgreSQL database. Expose database schema with filters and make PostgreSQL views for complex API endpoints. If you often write CRUD API for database tables manually, it will save you some time.
And more less popular projects
Like hadolint: Dockerfile linter, validate inline bash, written in Haskell; Purescript, Unison, Idris languages implemented in Haskell. I recommend to check ideas behinds those projects. Especially Unison's mind-blowing something naive idea about infinite computational resources.
Scala
Brings FP to Java land. Good path for whom looking for functional language to code and the best job opportunities. But not elegant language at all in my humble opinion.
spark
Stars: 40k
Apache Spark™ is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
Created in 2009 at UC Berkeley's AMPLab, open-sourced in 2010 - quite some time. De facto standard for big data processing making use of Scala's functionality. Adopted by Netflix, Uber, Pinterest, Alibaba.
Enso
Stars: 7k Interesting analytical tool and language with one-to-one visual counterpart. Its IDE has some bugs, and requiring login for some reason, but it is worth checking. And can be used for "Detecting Credit Card Fraud with Enso" as stated in their blog post.
Python with returns lib
Python is not functional language itself. And as I can relate does not want to be: look at lambda for instance. But this package allows to write more robust applications. Link to lib: GitHub - dry-python/returns: Make your functions return something meaningful, typed, and safe!
In closing
It is safe to say that functional language can be used in almost any domain excluding maybe system engineering, if not counting Rust as functional language. You may wonder after all why then functional programming languages are not dominating everywhere if it is so good? So Richard gave a good answer to it: Why Isn't Functional Programming the Norm? – Richard Feldman - YouTube. Let it brew.
Do you want full stack type safe scalable app?
Look closer at Gleam or Phoenix. It has good balance of simplicity and FP feature along side with good development experience. Maybe add CouchDB since it runs on BEAM too. Great article on using Gleam for a long time.
Do you want a queue?
RabbitMQ for general use, EMQX for IoT.
Do you want analytics?
Give Enso or MetaBase a shot.
Do you want complex reliable backend for WEB or for CLI?
Look at Haskell or OCaml. Which one specifically? There is a good article comparing from an experienced developer in both languages.
Do you want frontend?
Elm or Elm inspired frameworks. Fortunately there are infinitely many Elm inspired frameworks like mentioned Lustre or something esoteric like Unison lang library.
Do you want to try FP in TS or Python?
fp-ts for TypeScript, returns for Python.
Do you want insane Data/ML stuff?
Spark
P.S. Radar version of this article would be good.
References
Subscribe to my newsletter
Read articles from Roman M directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Roman M
Roman M
Backend developer, FP enjoyer