VPS Errors on Random Websites

Suggestive of problems with virtual private servers (a virtual machine hosted in a DC, with a dedicated operating system- typically Linux or Windows, root/admin acces and the ability to run websites, databases, APIs or background services).

AI Orchestrators

LangChain, CrewAI, AutoGen, ADK and LangGraph are examples of tools used for agent orchestration.

RAG with Pinecone and Chroma

Pinecone (pinecone.io) is "the knowledge engine for agents" - aka. a vector database for RAG pipelines.

Chroma is open source search infra for AI - also built on vector database technology.

Prompt Optimization - Worth the Investment

Prompt optimization involves trying various prompts to find the most effective prompt for an LLM. 

This kind of A/B testing, or A/B/C/D.. testing is going to burn tokens. It may be justified in some regulated domains where you need to show consistency.

The ideology is that even if you express your intent well, the LLM underneath may not produce the best output with your input. This is because (it is said) "being clear" is not the same as "being optimal" for a generative model.

AutoPrompt is one option for what it refers to as "prompt tuning".

DSpy (a Python framework for building AI systems) has also been used for the same use case.

A video on DSpy can be found here.

Generative Models - GANs/VAEs and Diffusion Models - O My

 GANs are obviously generative adversarial networks. 

• Developed by Ian Goodfellow and colleagues in June 2014
• In this construct, two neural networks compete with each other in the form of a zero-sum game, where one agent's again is the other agent's loss
• The concept is that competition forces both to get better
• The game is to provide high realism output
• They are used where you need hi-fidelity synthetic data, realistic imagery
• They are not the dominant model for consumer generative AI
• The theory behind GANs is an interesting application of probability spaces

Diffusion models.

• Class of latent variable generative models
• Based on diffusion processes in applied probability
• The goal of the model is the learn the diffusion process that underpin an image (strange concept in itself, and one that puts this technique in the realm of latent variable generative models)
• Two components - the forward diffusion process and the reverse sampling process
• Simple example of a diffusion process is ink dropped in water, droplets diffuse through the water
• One example application is denoising images (where image is blurred with Gaussian noise)
• Stable Diffusion and DALL-E are diffusion based image generators

VAEs are variational auto-encoders

• A variational auto-encoder (VAE) is an artificial neural network introduced by Kingma and Welling in 2013
• It is part of the families of probabilistic graphical models and variational Bayesian methods

** classes of model **

Latent variable generative models.

• Statistical model that relates a set of observable variables (also called manifest variables, indicators) to a set of latent variables
• Latent variables are those that can be observed /inferred only via a mathematical model
• They may correspond to aspects of physical reality
• Earliest reference: Francis Bacon, Novum Organum
• It may reduce the dimensionality of the data

Auto-encoder.

• Form of artificial neural network.
• Used to learn efficient codings of unlabeled data (unsupervised learning)
• An autoencoder learns two things:
• Encoding function - transforms the input data 
• Decoding function - that recreates the input data from the encoding/encoded representation
• Autoencoder learns an efficient representation (encoding) for a set of data, typically for dimensionality reduction, to generate lower dimensional embeddings for subsequent use by machine learning algorithms

Variations of auto-encoders that embody useful properties.

• Example: regularized auto-encoder (aka sparse, denoising and contractive autoencoders)
• Example: variational auto-encoder (used for generative applications)

"A continued process which... escapes the observation of the senses" - Francis Bacon

xcopy still works

You can still do a mean recursive copy in cmd.exe using:

xcopy *.* /s

But when copying to a target directory, make sure the directory exists, or it will create a file (as it has done consistently in previous versions).  

MS-DOS is still alive and well.

The Codex Incompleteness Theorem

A few Codex gaps with workarounds.

Codex will create a .git directory - but without initializing. Before you check in code, do a git init.

For Windows, IDE based builds won't work - as settings need to be done in the GUI to persist in config. This makes debugging from the IDE difficult - "if I change settings in IDE for my ease of debugging will it break build process used by Codex?"