GPT-5.6 Sol - Preview

Sol is OpenAI's strongest model yet.

It has a robust safety stack, including protections for higher-risk activity and repeated misuse.

A limited preview is being done with trusted partners first.

Claude's "Access Consciousness"& Rise of the "J-Space"

Anthropic's Claude model series has claimed "access consciousness" via the J-Space.

The Claim of the "J Space"; And the Parallel with Global Workspace Theory of Neuroscience

This claim has been presented as "A global workspace in language models" described at a high level here, and deep-dived in the paper  Verbalizable Representations Form a Global Workspace in Language Models (July 6, 2026, Wes Gurnee, Nicholas Sonofriew, Jack Lindsey et al). 

From the paper - it is clear that the analysis identifies "data structures of the mind" - where we replace with "mind" with "model" to get a window in on the model's thinking:

"we observe that language models maintain a privileged set of internal representations, available for report, modulation, and flexible internal reasoning, atop a much larger volume of automatic processing. We identify these representations using a new interpretability technique, which surfaces the concepts a model is poised to verbalize at any point in its processing".

What is interesting is the discovery of this so-called "J-Space" but also the interpretability technique. It offers a new way to "commune" with LLMs.

The phenomenon of "access consciousness" is described in the paper, a concept from behavioural and brain science. This is introduced as a purely functional notion - it's purported purpose is utilitarian, and not linked to subjective experience (sometimes called phenomenal consciousness).

Neuroscience has a global workspace theory where a "data structure" can be posted to the brain's "working set area" for use in reasoning and reporting.

The Evidence

The paper poses the question whether functional properties of a global workspace have emerged in LLMs.  It portrays the LLM's thinking as a plethora of vector representations, some constituting low-level bookkeeping and some embodying higher level ideas like "Golden Gate Bridge" or even emotions.  If such a workspace were to exist, we would expect a subtset of vectors to be prominently and preferentially present in the LLM's memory.

The Test for Presence

Verbal report - when asked what it is thinking about, LLM names concepts from its workspace

LiteLLM - Gateway to 100+ LLMs

LiteLLM is an open source library that lets you call into over 100 LLMs.

One aspect of calling into lots of LLMs is you have lots of API keys. These are stored in environment variables like OPENAI_API_KEY, ANTHROPIC_API_KEY - you can also feed these as direct arguments in code to LiteLLM.

You can also integrate a secrets manager like AWS Secrets Manager, Azure Key Vault and Google Secret Manager.

JetBrains Releases Mellum2

JetBrains (the home of IntelliJ and PyCharm) has released its Mellum2 Mixture-of-Experts coding model, of 12B parameters. 

The model is available under the Apache 2.0 license. 

The model has been published on Hugging Face and can be run locally.

A mixture-of-experts model works via a "gating network" which delegates work to smaller neural networks, the "experts", optimising overall performance. This model also leads to "sparse activation" - which means of all the possible parameters utilised by the model, only a subset are used per input.

Training an MoE model requires training the gating network and training the various "experts".

OpenCode and GLM Models

OpenCode is an open source AI coding agent. It connects to free models and also allows connection to commercial models. GLM models are supported.

LoRA in Real Workflows

LoRA, or low-rank adaptation, is a fine-tuning technique for LLMs (one of many disparate techniques). 

The idea is to inject low rank matrices into large pre training models.

Recall that the rank of a matrix A is the dimension of the vector space spanned by its columns. This in turn corresponds to the number of linearly independent columns of A.

So LoRA is essentially a dimensionality reduction of the column space of parameters to ease off compute.

Books and Resources on AI Engineering

Apart from staying up to date through websites there are a number of good books on AI Engineering. Here is a recommended reading list.

AI Engineering, Chip Huyen (2025, O'Reilly) - really good book on building systems on top of LLMs. Chip's Github is here.

Hands-On Large Language Models, by Jay Alammar and Maarten Grootendorst (O'Reilly) - uses Python to convey an understanding of how LLMs operate under the hood, covers similar ground to AI Engineering - definitely worth reading. It has quite a few text processing canned examples which are quite interesting.

Mathematics for Machine Learning, by Deisenroth et al. - not as directly connected to AI Engineering but good at explaining some of the underlying maths of ML intuitively (and in somewhat long winded fashion - at least from an engineering perspective).

GGUF = GPT-Generated Unified Format

GGUF is a special model file format used to run LLMs efficiently on your own computer. It stands for GPT-Generated Unified Format. It is the successor to GGML. Privacy-sensitive workflows are one application of these optimized-for-local execution model packages.

LiteLLM

LiteLLM lets you make calls into a variety of LLMs using OpenAI conventions. Think of it as a lightweight model router. LangChain is somewhat similar - but with a bunch of added stuff on agents and orchestration.

How Claude Code Works

Read more here.

Claude Code is an agentic assistant running in your terminal. It is an "agentic harness" around Claude.

What is Pydantic?

Pydantic is used in a number of Python frameworks and libraries - for example, it is used in Langchain extensively.

Pydantic is a widely used data validation library.  

It makes extensive use of the annotations feature in Python. It is worthwhile to understand them in the context of type hints.

Microsoft Launch Fara-7B: A CUA (Computer Use Agent) in SLM Form

And here we have it. Ready for action on Hugging Face, Sir.

Sandboxing and monitoring are recommended. The agent itself is a wrapper around Playwright.

IBM's Guide to Small Language Models

IBM have made a guide to SLMs.

Examples SLMs listed as:

• DistilBERT (DistilBERT is Google's groundbreaking BERT model in "distilled" form (hence the name "Distilled BERT"), retaining 97% of BERT's NLU abilities)
• Gemma
• GPT-4o mini
• Granite
• Llama
• Ministral
• Phi

Getting Jiggy with gpt-oss-20b (and why open weights matter)

gpt-oss-20b is an open weight language model. These so-called "open weights" reflect the pre-training the model has received.

The model is a significant 12GB download.

Microsoft Semantic Kernel

Microsoft Semantic Kernel is a "lighweight, open-source development kit" to build AI agents and integrate models into C#,  Python and Java code.

When you load up SK into a fresh Visual Studio Code (no extensions) it will prompt to install recommended extensions. These will include:

• ESLint - integrates ESLint JavaScript into VS Code (for static analysis)
  
• Prettier - integrates Prettier, the opinionated code formatter (for JavaScript, TypeScript and other webby stuff)
• Azure Functions - to quickly manage serverless apps directly from VS Code
• vscode-pdf - to display pdf files in VSCode (required to open PDF code maps for .NET and Python)

Cost Effective Deployment of Language Models

Cost effective deployment of language models (explicit financial as well as implicit environmental cost) is partly responsible for triggering the interest in small language models (SLMs) as alternatives for specific applications. 

Nvidia Research have a great paper on this entitled "Small Language Models are the Future of Agentic AI" with the recommendation that more routine tasks (non reasoning tasks) move from LLMs to SLMs. 

Fine tuning these SLMs for specific tasks can also enhance the effectiveness of deployed models.

Prompt Maintenance

So-called prompt engineering is fraught with execution risk. New features are added to LLMs all the time. What worked yesterday may not work (as well) today and may need to be further "tuned".

This high touch maintenance requirement may trigger a shift away from usage of prompts in production workflows to more truly engineering-oriented solutions that have better stability and resilience properties.

OpenAI Releases Open Models under Apache 2.0

OpenAI has released two open models as part of that is now dubbed the gpt-oss series.

• gpt-oss-120b
• gpt-oss-20b

The first is more suited to data centers, the latter is more for laptop and personal use.

They are both trained on the harmony response format (if going through a provider like HuggingFace, Ollama or vLLM you don't need to worry about it - the provider will deal with the format).

LLM Training Data

LLMs are trained on large data sets.  One such data set is Common Crawl which consists of 250 billion Internet pages with 3-5 billion pages added each month.  This is petabytes worth of data (1 petabyte = 10^15 bytes of digital information). The data is stored on Amazon's S3 service allowing direct download or access for Map-Reduce processing in EC2.

What is InstructLab?

Instruct Lab was developed by IBM Research and Red Hat and is an open source product. 

It is designed to improve training of LLMs (specifically reducing cost of training).  A basic intro can be found here. 

Instruct Lab uses fine-tuning (both knowledge tuning and skills tuning).