Lisp-Stat

Quick Plot

Mon, 16 Feb 2026 00:00:00 +0000

Overview

The geom and gg packages provide a set of composable helper functions for building Vega-Lite plot specifications. Inspired by R’s ggplot2, specifications are constructed by combining independent layers — geometry, labels, scales, coordinates and themes — rather than writing monolithic JSON-like plists by hand.

Each helper returns a plist. The function merge-plists recursively merges them into a single Vega-Lite spec, which vega:defplot compiles and plot:plot renders.

Package setup

The examples below assume you are working in the LS-USER package so you need to use the qualified forms (geom:point, gg:label, etc.) or import the appropriate symbols into the current package (which should be LS-USER):

Array Operations

Fri, 01 Aug 2025 00:00:00 +0000

Overview

The array-operations system contains a collection of functions and macros for manipulating Common Lisp arrays and performing numerical calculations with them.

Array-operations is a ‘generic’ way of operating on array like data structures. Several aops functions have been implemented for data-frame. For those that haven’t, you can transform arrays to data frames using the df:matrix-df function, and a data-frame to an array using df:as-array. This make it convenient to work with the data sets using either system.

Installation

Sat, 18 Jun 2022 00:00:00 +0000

Intallation methods

Online Notebook

The easiest way to get started is with the link above which will open a preconfigured notebook on mybinder.org.

Local Jupyter Container

You can also run the Jupyter notebook in a pre-built OCI image. This is a minimal Docker file:

FROM ghcr.io/lisp-stat/cl-jupyter:latest

Our images are based on Jupyter Docker Stacks and all of their documentation is applicable to the cl-jupyter image.

For a quickstart:

docker run -it -p 8888:8888 ghcr.io/lisp-stat/cl-jupyter:latest

# Entered start.sh with args: jupyter lab

# ...

#     To access the server, open this file in a browser:
#         file:///home/jovyan/.local/share/jupyter/runtime/jpserver-7-open.html
#     Or copy and paste one of these URLs:
#         http://eca4aa01751c:8888/lab?token=d4ac9278f5f5388e88097a3a8ebbe9401be206cfa0b83099
#         http://127.0.0.1:8888/lab?token=d4ac9278f5f5388e88097a3a8ebbe9401be206cfa0b83099

This command pulls the latest cl-jupyter image from ghcr.io if it is not already present on the local host. It then starts a container running a Jupyter Server with the JupyterLab frontend and exposes the server on host port 8888. The server logs appear in the terminal and include a URL to the server.

Basics

Sat, 20 Feb 2021 00:00:00 +0000

Preface

This document is intended to be a tutorial introduction to the basics of LISP-STAT and is based on the original tutorial for XLISP-STAT written by Luke Tierney, updated for Common Lisp and the 2026 implementation of LISP-STAT.

LISP-STAT is a statistical environment built on top of the Common Lisp general purpose programming language. The first three sections contain the information you will need to do elementary statistical calculations and plotting. The fourth section introduces some additional methods for generating and modifying data. The fifth section describes some features of the user interface that may be helpful. The remaining sections deal with more advanced topics, such as interactive plots, regression models, and writing your own functions. All sections are organized around examples, and most contain some suggested exercises for the reader.

Distributions

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/IPS9/Chapter%201%20Looking%20at%20Data.html

What is Lisp-Stat?

Mon, 01 Jan 0001 00:00:00 +0000

Lisp-Stat is a domain specific language (DSL) for statistical analysis and machine learning. It is targeted at statistics practitioners with little or no experience in programming.

Lisp has a history of being deployed for domain experts to use, and it’s a great language for beginners; the Symbolics Graphics Division wrote the software used by graphic artists to develop scenes in several films prior the rise of Pixar. One of the first statistical systems developed, XLisp-Stat, was a contemporary to R until the primary author joined the ‘R Core’ group.

Data Frame

Sun, 08 Mar 2026 00:00:00 +0000

Overview

A common lisp data frame is a collection of observations of sample variables that shares many of the properties of arrays and lists. By design it can be manipulated using the same mechanisms used to manipulate lisp arrays. This allow you to, for example, transform a data frame into an array and use array-operations to manipulate it, and then turn it into a data frame again to use in modeling or plotting.

Data Frame

Sat, 18 Jun 2022 00:00:00 +0000

Load data frame

(ql:quickload :data-frame)

Load data

We will use one of the example data sets from R, mtcars, for these examples. First, switch into the Lisp-Stat package:

(in-package :ls-user)

Now load the data:

(data :mtcars-example)
;; WARNING: Missing column name was filled in
;; T

Examine data

Lisp-Stat’s printing system is integrated with the Common Lisp Pretty Printing facility. To control aspects of printing, you can use the built in lisp pretty printing configuration system. By default Lisp-Stat sets *print-pretty* to nil.

Site Organisation

Sat, 18 Jun 2022 00:00:00 +0000

This manual is organised by audience. The overview and getting started sections are applicable to all users. Other sections are focused on statistical practitioners, developers or users new to Common Lisp.

Examples

This part of the documentation contains worked examples of statistical analysis and plotting. It has less explanatory material, and more worked examples of code than other sections. If you have a common use-case and want to know how to solve it, look here.

Data Relationships

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/IPS9/Chapter%202%20Data%20Relationships.html

Distributions

Sun, 28 Aug 2022 00:00:00 +0000

Overview

The Distributions system provides a collection of probability distributions and related functions such as:

Sampling from distributions
Moments (e.g mean, variance, skewness, and kurtosis), entropy, and other properties
Probability density/mass functions (pdf) and their logarithm (logpdf)
Moment-generating functions and characteristic functions
Maximum likelihood estimation
Distribution composition and derived distributions

Getting Started

Load the distributions system with (asdf:load-system :distributions) and the plot system with (asdf:load-system :plot/vega). Now generate a sequence of 1000 samples drawn from the standard normal distribution:

Getting Help

Sat, 18 Jun 2022 00:00:00 +0000

There are several ways to get help with Lisp-Stat and your statistical analysis. This section describes way to get help with your data objects, with Lisp-Stat commands to process them, and with Common Lisp.

Search

We use the algolia search engine to index the site. This search engine is specialised to work well with documentation websites like this one. If you’re looking for something and can’t find it in the navigation panes, use the search box:

Plotting

Sat, 18 Jun 2022 00:00:00 +0000

Overview

The plot system provides a way to generate specifications for plotting applications. Examples of plotting packages include gnuplot, plotly and vega/vega-lite.

Plot includes a back end for Vega-Lite; this tutorial will teach you how to encode Vega-Lite plot specifications using Common Lisp. For help on Vega-Lite, see the Vega-Lite tutorials.

For the most part, you can transcribe a Vega-Lite specification directly into Common Lisp and adapt it for your own plots.

Preliminaries

Load Vega-Lite

Load Vega-Lite and network libraries:

Common Lisp References

Sun, 07 Mar 2021 00:00:00 +0000

Numeric Utilities

Fri, 08 Aug 2025 00:00:00 +0000

Arithmetic

The arithmetic package provides a collection of mathematical and arithmetic functions for Common Lisp. These utilities are designed to simplify common numerical computations while maintaining efficiency through inline declarations where appropriate.

Basic Operations

same-sign-p

(same-sign-p &rest numbers)

Tests whether all arguments have the same sign (i.e., all are positive, negative, or zero).

(same-sign-p 1 2 3)    ; ⇒ T
(same-sign-p 1 -2 3)   ; ⇒ NIL
(same-sign-p -1 -2 -3) ; ⇒ T

square

(square x)

Returns the square of a number. This function is inlined for performance.

Linear Algebra

Fri, 08 Aug 2025 00:00:00 +0000

Overview

LLA (Lisp Linear Algebra) is a high-level Common Lisp library for numerical linear algebra operations. It provides a Lisp-friendly interface to BLAS and LAPACK libraries, allowing you to work with matrices and vectors using Lisp’s native array types while leveraging the performance of optimized numerical libraries.

The library is designed to work with dense matrices (rank-2 arrays) containing numerical values. While categorical variables can be integer-coded if needed, LLA is primarily intended for continuous numerical data.

Vega Plotting

Mon, 26 Apr 2021 00:00:00 +0000

The plots here show equivalents to the Vega-Lite example gallery. Before you begin working with these example, be certain to read the plotting tutorial where you will learn the basics of working with plot specifications and data.

Preliminaries

Load Vega-Lite

Load Vega-Lite and network libraries:

(asdf:load-system :plot/vega)

and change to the Lisp-Stat user package:

(in-package :ls-user)

Load example data

The examples in this section use the vega-lite data sets. Load them all now:

(vega:load-vega-examples)

Bar charts

Bar charts are used to display information about categorical variables.

Your First Project

Sat, 18 Jun 2022 00:00:00 +0000

Lisp-Stat includes a project template that you can use as a guide for your own projects.

Use the template

To get started, go to the project template

Click Use this template
Select a name for your new project and click Create repository from template
Make your own local working copy of your new repo using git clone, replacing https://github.com/me/example.git with your repo’s URL: git clone --depth 1 https://github.com/me/example.git
You can now edit your own versions of the project’s source files.

This will clone the project template into your own github repository so you can begin adding your own files to it.

Select

Sun, 07 Mar 2021 00:00:00 +0000

Overview

Select provides:

An API for taking slices (elements selected by the Cartesian product of vectors of subscripts for each axis) of array-like objects. The most important function is select. Unless you want to define additional methods for select, this is pretty much all you need from this library. See the API reference for additional details.
An extensible DSL for selecting a subset of valid subscripts. This is useful if, for example, you want to resolve column names in a data frame in your implementation of select.
A set of utility functions for traversing selections in array-like objects.

It combines the functionality of dplyr’s slice, select and sample methods.

SQLDF

Sun, 07 Mar 2021 00:00:00 +0000

Overview

sqldf is a library for querying data in a data-frame using SQL, optimised for memory consumption. Any query that can be done in SQL can also be done in the API, but since SQL is widely known, many developers find it more convenient to use.

To use SQL to query a data frame, the developer uses the sqldf function, using the data frame name (converted to SQL identifier format) in place of the table name. sqldf will automatically create an in-memory SQLite database, copy the contents of the data frame to it, perform the query, return the results as a new data frame and delete the database. We have tested this with data frames of 350K rows and there is no noticeable difference in performance compared to API based queries.

Statistics

Sat, 14 Mar 2026 00:00:00 +0000

Overview

The statistics system is organised in three layers.

stat-generics defines the shared generic functions — mean, variance, standard-deviation, quantile, quantiles, median, mode, and modes — together with the weight type hierarchy. Neither package contains standalone implementations; they provide the protocol that other implementation packages specialise.

batch-statistics provides methods on those generics specialised on fully realised Common Lisp sequences, together with additional descriptive functions that have no natural streaming equivalent.

Special Functions

Sun, 07 Mar 2021 00:00:00 +0000

The library assumes working with 64 bit double-floats. It will probably work with single-float as well. Whilst we would prefer to implement the complex domain, the majority of the sources do not. Tabled below are the special function implementations and their source. This library has a focus on high accuracy double-float calculations using the latest algorithms.

function	source
erf	libm
erfc	libm
inverse-erf	Boost
inverse-erfc	Boost
log-gamma	libm
gamma	Cephes
incomplete-gamma	Boost

Error rates

The following table shows the peak and mean errors using Boost test data. Tests run on MS Windows 10 with SBCL 2.0.10. Boost results taken from the Boost error function, inverse error function and log-gamma pages.

Code Repository

Mon, 01 Jan 0001 00:00:00 +0000

Below is a partial list of the consolidated XLispStat packages from UCLA and CMU repositories. There is a great deal more XLispStat code available that was not submitted to these archives, and a search for an algorithm or technique that includes the term “xlispstat” will often turn up interesting results.

Artificial Intelligence

Genetic Programming

Cerebrum: A Framework for the Genetic Programming of Neural Networks. Peter Dudey. No license specified.
[Docs]
GAL: Functions useful for experimentation in Genetic Algorithms. It is hopefully compatible with Lucid Common Lisp (also known as Sun Common Lisp). The implementation is a “standard” GA, similar to Grefenstette’s work. Baker’s SUS selection algorithm is employed, 2 point crossover is maintained at 60%, and mutation is very low. Selection is based on proportional fitness. This GA uses generations. It is also important to note that this GA maximizes. William M. Spears. “Permission is hereby granted to copy all or any part of this program for free distribution, however this header is required on all copies.”
mGA: A Common Lisp Implementation of a Messy Genetic Algorithm. No license specified.
[Docs, errata]

Machine Learning

Machine Learning: Common Lisp files for various standard inductive learning algorithms that all use the same basic data format and same interface. It also includes automatic testing software for running learning curves that compare multiple systems and utilities for plotting and statistically evaluating the results. Included are:

AQ: Early DNF learner.
Backprop: The standard multi-layer neural-net learning method.
Bayes Indp: Simple naive or “idiot’s” Bayesian classifier.
Cobweb: A probabilistic clustering system.
Foil: A first-order Horn-clause learner (Prolog and Lisp versions).
ID3: Decision tree learner with a number of features.
KNN: K nearest neighbor (instance-based) algorithm.
Perceptron: Early one-layer neural-net algorithm.
PFOIL: Propositional version of FOIL for learning DNF.
PFOIL-CNF: Propositional version of FOIL for learning CNF.

Raymond J. Mooney. “This program may be freely copied, used, or modified provided that this copyright notice is included in each copy of this code and parts thereof.”

Contributor License Agreement

Mon, 01 Jan 0001 00:00:00 +0000

First, if you are contributing on behalf of your employer, ensure you have signed a contributor license agreement. Then follow these steps for contributing to Lisp-Stat:

Step 1: Get source code.
Step 2: Get approval and modify the source code.
Step 3: Get your code reviewed and committed to the project.

You may also be interested in the additional information at the end of this document.

Contributor License Agreement

Contributor License Agreements (CLAs) are common and accepted in open source projects. We all wish for Lisp-Stat to be used and distributed as widely as possible, and for its users to be confident about the origins and continuing existence of the code. The CLA help us achieve that goal. Although common, many in the Lisp community are unaware of CLAs or their importance.

Contributing to Documentation

Mon, 01 Jan 0001 00:00:00 +0000

Creating and updating documentation is a great way to learn. You will not only become more familiar with Common Lisp, you have a chance to investigate the internals of all parts of a statistical system.

We use Hugo to format and generate the website, the Docsy theme for styling and site structure, and Netlify to manage the deployment of the documentation site (what you are reading now). Hugo is an open-source static site generator that provides us with templates, content organisation in a standard directory structure, and a website generation engine. You write the pages in Markdown (or HTML if you want), and Hugo wraps them up into a website.

Contribution Ideas

Mon, 01 Jan 0001 00:00:00 +0000

Special Functions

The functions underlying the statistical distributions require skills in numerical programming. If you like being ‘close to the metal’, this is a good area for contributions. Suitable for medium-advanced level programmers. In particular we need implementations of:

gamma
incomplete gamma (upper & lower)
inverse incomplete gamma

This work is partially complete and makes a good starting point for someone who wants to make a substantial contribution.

Documentation

Better and more documentation is always welcome, and a great way to learn. Suitable for beginners to Common Lisp or statistics.

Environment Setup

Mon, 01 Jan 0001 00:00:00 +0000

The first thing you’ll need to decide is what IDE you want to use. Traditionally Common Lisp developers use emacs and a connection to lisp like slime/swank for an interactive experience. However emacs, whilst powerful, presents a steep learning curve, so if you prefer a GUI based IDE you should start there. For detailed instructions, see the following from the cl-cookbook:

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/array-operations

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/data-frame

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/dfio

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/cephes.cl/

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/lla

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/linear-algebra/

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/lisp-stat

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/numerical-utilities/

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/select

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/special-functions

Mon, 01 Jan 0001 00:00:00 +0000

Your browser can not display embedded frames. You can access the embedded page via the following link: https://lisp-stat.github.io/statistics

Getting Started

Mon, 09 Mar 2026 00:00:00 +0000

It’s never been easy for a developer to get started in Common Lisp. Emacs, though a powerful editor, isn’t considered an IDE by modern standards. Setting up a compiler, quicklisp, slime, swank, and then learning an entirely new programming paradigm has scared off many would-be entrants to the Common Lisp community.

Given the size of the Common Lisp community this is understandable. Making the new user experience smooth and frictionless as possible is hard work. It’s the kind of work that no one volunteers for; it’s the kind of work you have to be paid for.

Introducing Quick Plot: ggplot-Style Plotting for Lisp-Stat

Sat, 21 Feb 2026 00:00:00 +0000

Today we’re releasing quick-plot, a new system for Lisp-Stat that bring ggplot2-style composable plotting to Common Lisp. plot/vega is able to render any Vega-Lite specification (here’s 50 examples), but it can at times be a bit wordy constructing the nested plists by hand.

Overview

Instead of specifying each plot in detail, quick-plot provides a set of helper functions that work similarly to geom in ggplot. Essentially we borrow ggplot’s concept of a plot as a composition of independent layers (which is what Vega-Lite does anyway), each with one concern:

2023 End of Year Summary

Fri, 29 Dec 2023 00:00:00 +0000

We started the year by working through the examples from the first chapter of the Introduction to the Practice of Statistics. Whilst this is an excellent test for Lisp-Stat, Chapter 2 presented some major challanges to the plotting system, so the highlight for this year is Plot 2.0. This is a breaking change, however the fix for plots is relatively simple.

The updates for 2023 include:

IPS Chapter 1 complete
IPS Chapter 2 plotting
Sampling data frames
Adopted cl-gists so we can work with the Vega eco-system on Github
Added Lisp Linear Algebra, CFFI wrappers for BLAS and LAPACK
Improved tutorials, examples and documentation
Plot 2.0
- Smoothers for non-parametric regression
- Plot helpers for qq-plots and scatterplots
- New data sources: named data sets, URLs, embedded
- Publish plots to Github gists
- Edit plots in Vega Edit online editor
- Fully support multi-layer, multi-data set plotting

Introduction to the Practice of Statistics

Sun, 05 Feb 2023 00:00:00 +0000

The examples from the first chapter of the book are now completed. This chapter focuses mostly on graphics, and the examples here supplement those in the plotting examples.

2022 Release

Tue, 19 Jul 2022 00:00:00 +0000

Overview

This release is the third major refactoring of Lisp-Stat, with particular emphasis on plotting. I have attempted to make plotting as easy in Lisp-Stat as it is in Vega-Lite, upon which it is based. In addition, much clean-up and polishing of sharp edges has taken place as a result of using Lisp-Stat on real-world projects.

As always, comments and contributions are welcome.

What’s New

Lisp-Stat

Data loading functions akin to R’s data function. You can now load the Lisp-Stat data sets with this function, e.g. (data :mtcars). The mechanism is extensible for add on packages to use as well.
Additional cleansed and annotated data sets included.

Plot

The plotting system has been completely reworked. For example, to create a scatter plot of horsepower vs. miles per gallon:

(asdf:load-system :plot/vega)
(data :vgcars)
(plot:plot
  (vega:defplot hp-mpg
  `(:title "Horsepower vs. MPG"
    :data (:values ,vgcars)
    :mark :point
	:encoding (:x (:field vgcars:horsepower)
	           :y (:field vgcars:miles-per-gallon)))))

50+ new plotting examples for commonly used statistical plots, along with a plotting tutorial.

Data-frame

Paid technical debt, refactored, made ready for future progress.
Added filter-rows, remove-columns, remove-columns! and rename-column!.
Improved unit tests.
Added column types: categorical (factor) and temporal.
Improved error handling, conditions and restarts.
Working prototypes for stack, split-apply-combine, sort operations.

Array Operations

Array operations has been rehomed and now lives in the Lisp-Stat github organisation. Significant improvements to documentation over the previous repository.

Statistical Analysis with Lisp-Stat

Sun, 09 May 2021 00:00:00 +0000

Lisp-Stat is the culmination of many months work to pull together the best-in-class statistical analysis packages available in Common Lisp, under a commercially friendly license, usable ‘out of the box’. It is an environment for statistical computing, conceptually similar to R, that is also suitable for front-line production deployments. It grew out of a desire to have an environment for rapidly prototyping analytical and A.I. solutions, and move directly to production environments with minimal friction. Typically there is a gap between the modeling environment, perhaps R or Python, and the delivery environment. Lisp-Stat closes that gap.

Data Protection Notice

Mon, 01 Jan 0001 00:00:00 +0000

This Data Protection Notice (“Notice”) sets out the basis which Symbolics Pte Ltd (“we”, “us”, or “our”) may collect, use, disclose or otherwise process personal data of our customers in accordance with the Personal Data Protection Act (“PDPA”). This Notice applies to personal data in our possession or under our control, including personal data in the possession of organisations which we have engaged to collect, use, disclose or process personal data for our purposes.

Personal Data

As used in this Notice: “customer” means an individual who (a) has contacted us through any means to find out more about any goods or services we provide, or (b) may, or has, entered into a contract with us for the supply of any goods or services by us; and “personal data” means data, whether true or not, about a customer who can be identified: (a) from that data; or (b) from that data and other information to which we have or are likely to have access.

Search Results

Mon, 01 Jan 0001 00:00:00 +0000