rel:: [[Programming Languages]] [[Clojure]] - Author:: [[Rich Hickey]] - Source:: https://download.clojure.org/papers/clojure-hopl-iv-final.pdf - Source:: https://clojure.org/about/history - Recommended By:: [[Stuart Halloway]] #### Source [[history_of_clojure.pdf]] # Ideas - [[excluding features is a key design decision]] - "I am certain that had [[Clojure]] had [[reader macros]], and users availed themselves of them, the large, composable, data-driven library ecosystem that arose around [[Clojure]] would have been compromised or thwarted" - [[tight feedback loops are key to quality at speed]] - [[REPL]] - [[zettle/understandable programs have fewer defects]] - [[immutable data]] - [[persistent data structures]] - [[pure functions]] - [[data as simple abstraction]] - "These facilities make [[Clojure]] a good language for information programming. Specifically, they make it easier to write context-independent, generic, loosely coupled information processing programs that can be driven by data. Keys and maps are first-class values not encoded in specific program types, readable from text without specific program code, and can be produced by different languages. This is the way that large, loosely coupled programs that interoperate over wires are constructed, and in my opinion it’s the way that large loosely coupled programs that don’t (yet) operate over wires should be constructed as well." - [EDN]([[Extensible Data Notation]]) - "In information systems large and small, ==one is often forced to abandon the class/record system and just start using maps==, which often have no syntax or other language support. And ==on the database side, one often starts using generic entity/attribute/value (triple) tables to get the flexible information support one needs==." - "At the same time, I had been doing some research into ==[[RDF]] [W3C 2014], a system designed around the needs of open, extensible information management==. There, the semantics of properties (attributes) are inherent in the attributes themselves, not determined by their presence in some named, fully enumerated aggregate/class/record type. Aggregation is ad hoc. ==Properties are named by URIs and thus conflict-free==. The [[RDF]] design resonated with me and was a big influence on both [[Clojure]] and [[Datomic]]." - [[backwards compatibility as key value for sustainability]] - "There were some deprecations and removals in 1.1, and this was pretty much the last of that. ==Moving forward, Clojure was to be extremely conservative about breaking changes. I wanted Clojure to be a stable tool for professionals, not a platform for experimentation.==" # Highlights - Clojure was designed to be a general-purpose, practical [[functional programming]] language, suitable for use by professionals wherever its host language, e.g., [[Java]], would be. Initially designed in 2005 and released in 2007, [[Clojure]] is a dialect of [[Lisp]], but is not a direct descendant of any prior [[Lisp]]. It complements programming with [[pure functions]] of [[immutable data]] with concurrency-safe state management constructs that support writing correct [multithreaded]([[multithreading]]) programs without the complexity of mutex locks. - Most of the ideas in [[Clojure]] were not novel, but their combination puts Clojure in a unique spot in language design ([[functional programming|functional]], hosted, [[Lisp]]). - When you take the broadest notion of Lisp, programming in and with fundamental data structures, Clojure is both clearly a Lisp and seeking to extend the Lisp idea. In being built on abstractions and strongly focusing on functional programming, it is a novel [[Lisp]]. - [[Clojure]] embodies and demonstrates the continued viability of the [[Lisp]] idea, including the importance of a simple core, direct use of data, extension via libraries, and dynamic typing in delivering flexibility and reducing coupling. - I wanted a language as acceptable as [[Java]] or C#, but supporting a much simpler [[programming model]], to use for the kinds of information system development I had been doing professionally. - The first [[Clojure]] compiler was a [[Common Lisp]] program that generated [[Java]] and C# code. - [[Clojure]] attributes - Code is Data - `read/print` - Small Core Language. - Hosted - [[Clojure]] is always compiled (on the [[JVM]], to [[Java]] bytecode), there is no interpreter. - [[Clojure]] compiler didn’t need even basic optimizations, e.g., dead code elimination. It is a simple hand-rolled recursive descent compiler, written in [[Java]]. - [[Clojure]]'s approach to being hosted was a significant factor in its adoption, according to user experience reports and survey results. Early adopters could ‘sneak in’ [[Clojure]] as just another [[Java]] library. In adopting [[Clojure]] they got an immediate increase in power, as [[Clojure]]'s functional library and [data structures]([[persistent data structures]]) were added to their toolkit without any loss. And the benefits accrue on an ongoing basis; as [[Java]] API support is a top priority for new services, [[Clojure]] users can access these services as soon as [[Java]] can. - Clojure and programs written in Clojure are Java libraries and bytecode when on the JVM, are .Net assemblies on the CLR, and are Javascript on Javascript engines. - it was never my intention that entire [[Clojure]] applications be portable between hosts. My objective was to support portability of the language, its libraries - [[Clojure]] host needs - [garbage collected]([[garbage collection]]) - some high performance mechanism and/or optimization for [[runtime polymorphic dispatch]]. - [Tangible at Runtime]([[Reflection]]) - [[REPL]] - Flexible - [[Lisp-1]] and [Macros]([[metaprogramming]]) - [[Clojure]] symbol resolution - uses a single resolution mechanism for symbols in the function position and elsewhere, and is thus a [[Lisp-1]] - [quasiquote]([[syntax-quote]]), called [[syntax-quote]] and designated by ` Syntax-quote will take a form containing symbols and replace any unqualified symbol (i.e. one without a namespace part) with a fully-qualified one, where the namespace portion corresponds to the mapping in the namespace where syntax-quote was called. - symbols are a simple immutable type akin to strings, with no storage. - [[macro hygiene problem]] - However, Clojure supports programmatic macros (arbitrary functions of data → data, such data including symbols) in the [[Common Lisp]] style, which has been considered to be problematic without the separate function space of [[Lisp-2]]. - in [[Common Lisp]] the conflation of symbols and storage locations, and the interning reader, were the sources of the conflict. - If one wants to introduce conflict-free local names Clojure has **gensym**. - Taken together, this has worked out to be a quite satisfactory recipe for avoiding the so-called [“hygiene problem”]([[macro hygiene problem]]) - I am certain that had [[Clojure]] had [[reader macros]], and users availed themselves of them, the large, composable, data-driven library ecosystem that arose around [[Clojure]] would have been compromised or thwarted - [[immutable data]] and [[persistent data structures]] - I found what I wanted in [[hash array mapped tries]] (HAMTs) [Bagwell 2001] - [[Clojure]]’s success with and evangelism of persistent [HAMT]([[hash array mapped tries]])s influenced their subsequent adoption by [[Scala]] (release 2.8, in 2010), [[Haskell]] (unordered containers, in 2011), [[Erlang]] (large maps, release 18, in 2015) and others. Bagwell and Rompf [2011] went on to refine Clojure’s approach to immutable vectors with RRB-Trees. - Clojure’s data structure support makes it possible to directly take on a problem using primarily the provided data structures and library. Such code is succinct (due to data literals), performant (persistent [HAMT]([[hash array mapped tries]])s have best-in-class performance for [persistent maps]([[persistent data structures]])), functional (due to immutability) and idiomatic (data literal support begets use of the libraries and the functional strategy). - key abstractions - seq abstraction - (eventually) [[transducers]] - In [[Clojure]] I eventually decided to support the categoric partitioning of collection types as per the design of Java’s collection library, with which I agree: collections are either sequential, sets or maps, equality is supported within but not between partitions, and anything more specific about the collection types is irrelevant for equality. Thus lists, seqs and vectors with the same elements in the same order are equal, but a set and vector with the same elements are not - [[Clojure]]’s rigorous yet polymorphic equality semantics mean that even for deeply nested heterogeneous data structures ‘=’ “just works”, an essential foundation for value-oriented functional programming. - Under the hood, protocols define Java interfaces, and deftype/defrecords that have inline protocol definitions establish an inheritance relationship. At an invocation point of a protocol, the compiler creates a call site with a fast path for interface implementations and a polymorphic inline cache for types externally extended to satisfy the protocol. In this way, if programmers use either inline definition in deftype or defrecord, or use reify, they get performance on par with Java’s native polymorphic dispatch, and otherwise the performance is still good. - deftype is like defrecord but does not generate support for the map abstraction, while it does support mutable fields etc. It is for bottom-level plumbing tasks. - [[Extensible]] records as values - In information systems large and small, ==one is often forced to abandon the class/record system and just start using maps==, which often have no syntax or other language support. And ==on the database side, one often starts using generic entity/attribute/value (triple) tables to get the flexible information support one needs==. - When combined with Java package name reverse-domain conventions (e.g., `:org.my-org/my-name`, as is encouraged in [[Clojure]]) keywords are as useful for defining openly extensible, mergeable, conflict-free keysets as are URIs for [[RDF]] properties. - At the same time, I had been doing some research into ==[[RDF]] [W3C 2014], a system designed around the needs of open, extensible information management==. There, the semantics of properties (attributes) are inherent in the attributes themselves, not determined by their presence in some named, fully enumerated aggregate/class/record type. Aggregation is ad hoc. ==Properties are named by URIs and thus conflict-free==. The [[RDF]] design resonated with me and was a big influence on both [[Clojure]] and [[Datomic]]. - Curious features - Taking on the design and implementation of an [STM]([[Software Transactional Memory]]) was a lot to add atop designing a programming language. In practice, the [STM]([[Software Transactional Memory]]) is rarely needed or used. It is quite common for [[Clojure]] programs to use only atoms for state, and even then only one or a handful of atoms in an entire program. But when a program needs coordinated state it really needs it, and without the [STM]([[Software Transactional Memory]]) I did not think [[Clojure]] would be fully practical. - Multimethods are quite flexible, and performance is good enough for many applications, e.g., multimethods are used for Clojure’s polymorphic printing. But they cannot be made as fast as Java method dispatch. Thus the initial release of [[Clojure]] lacked a sufficient [[polymorphic dispatch]] system to define something like [[Clojure in Clojure]]. - These facilities make [[Clojure]] a good language for information programming. Specifically, they make it easier to write context-independent, generic, loosely coupled information processing programs that can be driven by data. Keys and maps are first-class values not encoded in specific program types, readable from text without specific program code, and can be produced by different languages. This is the way that large, loosely coupled programs that interoperate over wires are constructed, and in my opinion it’s the way that large loosely coupled programs that don’t (yet) operate over wires should be constructed as well. - There were some deprecations and removals in 1.1, and this was pretty much the last of that. ==Moving forward, Clojure was to be extremely conservative about breaking changes. I wanted Clojure to be a stable tool for professionals, not a platform for experimentation.== - I should have considered more the experience of people who ‘get it wrong’ and the importance of good error messages in setting them right, and prioritized that early on. - Clojure distills important principles acquired from programming information systems in the large, especially those around achieving loose coupling via data-driven interfaces. This fosters a unique library ecosystem with a high degree of compatibility and composability, much higher than I’ve seen with library approaches that require the coordination of code-driven constructs.