EC2 instances have different specializations

Instance types are grouped under instance families. These include general purpose, compute optimized. memory optimized and others.

General purpose are a good starting point.

Compute optimized are good for machine learning (a compute intensive task). Gaming servers also would benefit from compute optimized EC2 instances, as would high performance computing and scientific applications.

Memory-optimized is good for applications that use large data sets in-memory. This differs from storage-optimized for workloads that utilise a great deal of locally stored data.

Accelerated-computing instances are good for floating point number calculations, graphics processing and pattern matching. They use hardware accelerators (like GPUs).

After choosing instance type, choose instance size. Performance and cost should be key paramaters here.

Statistical Similarity

KL divergence is a way to measure "closeness" of two probability distributions. It is used in the literature on model cloning/model distillation.

Another name for it is "relative entropy".

Its full form is Kullback-Leibler divergence. 

It is denoted D[KL]( P || Q ). P is the true probability distribution and Q is the approximating probability distribution.

Mathematically:

D[

Downsampling from a Data Science Perspective

Downsampling in data science and data processing is as follows (this excludes the DSP, or digital signal processing, technical definition of downsampling - which is similar in spirit but differently defined).

Downsampling involves reducing the number of data points in a data set to enable comparability (sometimes referred to as "balancing the data").  This helps machine learning models avoid bias towards a dominant class.

Various approaches to downsampling (e.g. random downsampling) are described in this IBM article.

Get Python SHAP Explainer in Ubuntu

pip install shap

is what you need.

It has dependencies on:

• numba
• scikit-learn

Numba is a very interesting tool. It is an open source JIT compiler that translates a subset of Python and NumPy code into fast machine code. It uses the LLVM compiler library.

scikit-learn is the de facto Python machine learning kit. Everyone knows scikit-learn! But knowing it well - ah that's the rub!

You may see llvmlite downloaded too as part of shap, to support Numba.

A More Technical Take on Logits

Mathematically, the logit is the inverse of the standard logistic function.

sigma(x) = 1 / (1+ exp(-x)) 

Note that the standard logistic function has only positive signs in it, the only hint of anything negative is the presence of the negative exponentation.

Hence logit is equal to

sigma^(-1)(p) = ln (p/ (1-p)) for p in (0,1).

Due to this, we sometimes call logit as log-odds, since it is the logarithm of the odds function p/(1-p) where p is probability. 

However, this definition is like getting the answer to something where you don't know the question.

Logits in TensorFlow

Logits refer to the vector of (non-normalized) predictions that a classification model generates, which is then normally passed to a normalization function.

See also this course.

Boosting versus Bagging

Boosting and bagging are two classes of machine learning techniques.

• Libraries for Boosting:  catboost, XGBoost (eXtreme Gradient Boosting)
• Libraries for Bagging: scikit-learn (BaggingClassifier, BaggingRegressor), imbalanced-learn (scikit-learn extension for imbalanced data), ML-Ensemble

Boosting is basically stacking mini-models that incrementally improve on previous models. Bagging is using ensemble/averaging techniques i.e. running models in parallel and computing some form of average.

Hacking Transformers with Hugging Face

Knowledge here. But in short:

pip install huggingface_hub

pip install --upgrade huggingface_hub

To test the install, you can try the below:

 python -c "import huggingface_hub; print(huggingface_hub.__version__)"

 python3 -c "import huggingface_hub; print(huggingface_hub.__version__)"

pip install transformers tensorflow (if you are using tensorflow, else type torch)

pip install transformers tensorflow datasets 

To test bring up Python CLI and do:

from transformers import AutoTokenizer.

The Runtime Formerly Known as TensorFlow Lite

LiteRT is the  Google on-device runtime for machine learning, formerly known as TensorFlow Lite. 

You can convert TensorFlow, PyTorch and JAX models to the TFLite format. 

This can be done using AI Edge conversion tools.

LiteRT rises to various ODML (On-Device Machine Learning) challenges:

1. Connectivity - ability to execute without an Internet connection

2. Size - reduced model and binary size

3. Privacy/data restrictions - no personal data leaves the device

4. Power consumption - efficient inference and a lack of network connections

Operationally, LiteRT models use an efficient portable format known as FlatBuffers, and the .tflite file extension. (See here for the difference between FlatBuffers and protobuf).

What is the "tensor" in TensorFlow?

In TensorFlow, a tensor is a multidimensional array used to represent data in a machine learning model. It generalizes scalars (0D-array), vectors (1D-array) and matrices (2D-array) to higher dimensions.

But don't be fooled. A true mathematical tensor has much more going on behind the scenes, being multilinear maps with specified transformation rules. 

A TensorFlow tensor is a looser construct than that.

Describing Einstein's General relativity mathematically involves the use of tensors of the math variety.

Deeper Look at Sequential Model Building in TensorFlow

Let's revisit the model building command in TensorFlow in our "hello world" equivalent example.

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10)
])

So Sequential lets you build up a model in layers (see Layer class, or tf.keras.Layer, that inherits from Operation). 

Layers are callable objects. In Python, a callable is any object that can be called using parentheses (optionally with arguments). Read the implementation here (in keras/source/layers/layer.py).

But what does the Flatten method/Layer do?

Dense creates a densely-connected NN Layer (convolutional neural network architecture). 

• The first argument is the positive integer units, representing the dimensionality of the output space
• The second argument is the activation function to use (if this is missing, no activation is applied which is actually linear activation a(x) = x)

Activation functions in a neural net introduces non-linearity into the network.  

• Essentially these functions work with neurons and transform neural computations into output signals
• ReLU (rectified linear unit) is one of the most widely used activation functions in neural networks (f(x) = max(0,x)).

Train your First Neural Network on the MNIST dataset

You can train your first neural network on the MNIST dataset (used for image recognition models). The MNIST example is tantamount to being a "hello world" of machine learning programs.

Key features:

• Use Keras API as a "portal" into TensorFlow library to build the neural network
• Use "Sequential" model - allows you to add "layers" sequentially
• "Feed" the model the training data (creating the model takes a bit of study/effort)
• Model gives back a vector of "Logits" or "Logs-odd" scores, one per class
• Run softmax to convert these scores to probabilities
• Compile the model - with an optimizer and a loss function, configure for 'accuracy'
• model.fit
• model.evaluate to see how the model performed (was it a good fit to the data)

Doing this example immediately raises a billion questions! Answering these questions will help you in future machine learning projects with TensorFlow. So get your answers now!

Some numbers to remember in this "post game analysis" are 0 to 255 and 28x28.

All About the Data - the MNSIT Dataset & (Numpy-friendly) Data Format 

The MNIST dataset consists of 60,000 training images of handwritten digits and 10,000 test images, each a size of 28x28 pixels. Images are grayscale and numbers are 0-9. The data set is vectorized and in numpy format. Each pixel has an encoding of 0 to 255 (typical for grayscale images) where the number represents brightness, 0 is black and 255 is white.

The Data Set Loading Process (Involves Normalization)

So MNIST is one of the built-in datasets in Keras. 

The first step is to normalize the data by dividing each pixel value (in the training and testing data set) by PIXEL_MAX=255 which creates a value between 0 and 1 (inclusive) and converts an integral value into a decimal value.

Model.fit - In depth

How does this from a function-calling perspective.

How do I see how good this model is visually?

This requires some additional programming.

Google Colab

Google Colaboratory ("Colab") is a hosted Jupyter notebook which includes free access to GPUs and TPUs. It is for machine learning, data science and education.

For cool datasets to explore ML with, check out Google Dataset Search.

There is an interesting Colab workbook by Ashwin Rao on the SVB crisis.

Colab supports a large number of constantly upgraded Python packages including kagglehub (to use Kaggle resources) and narwhals (dataframe library).

pip install tensorflow

This will install the current stable release for TensorFlow.

Keras Models API

 Keras Models API provides three ways to create models.

• Sequential Model - the simple model - consists of layers. applied in succession.  You can create a Sequential model by passing a list of layers to the constructor of Sequential.
• The alternative, and preferred method for most use cases, is the Functional API. which is more flexible than the keras.Sequential API.  It enables the building of graphs of Layers.
• Model subclassing is building from scratch. for out of the box use cases.

All Eyes on Keras - Layer = IO TRANSFORMATION

What is it and Why Use it

Keras is the high-level API for TensorFlow, covering all aspect of workflow, from data processing to (hyperparameter) tuning to deployment.  It is the API to be used by default.

 

Layers and Models, Layers and Models

The core data structs of Keras are layers and models.

• A layer is a simple input/output transformation
• A model is a directed acyclic graph (DAG) of layers (production flow of layers, and thus a production flow of transformations)

A model is thus a "special" series of input-output transformations i.e. a "series" of layers.

Sidebar: what are hyperparameters

Hyperparameters are parameters you set before training a model. They can be set by a user or by a tuning algorithm. Example parameters could include learning rate (how fast the model learns), or number of layers in the neural network (more layers the more complex patterns the neural net can learn).

Machine Unlearning

As a machine learns, so must it unlearn.  

This ability is needed if an LLM ingests copyrighted content or personal data - it must be able to unlearn information it is not permitted to have. This could also apply to fallacious or untrusted data.

IBM in an article have noted the lack of industry wide tools to evaluated the effectiveness of unlearning.

The IBM piece also highlights research by Microsoft on machine unlearning. This also states the problem of the high cost of retraining models (this costly training process is what has spiked demand for GPUs).

A research paper, which styles itself as a "bridge" paper between unlearning research in classification models to unlearning in generative models focusing on the I2I (image-to-image) generation space.

In the IBM article, the writers go on to describe the SPUNGE framework they have developed for machine unlearning (SPUNGE being short for Split, Unlearn, Merge).

Papers with Code

Papers with Code is a Meta AI initiative that organizes machine learning papers under various themes including Computer Vision, Natural Language Processing, Reasoning, Time Series and Knowledge Representation. Some of these papers are written by corporate researchers contributing to open source.

Dude, FP16, really? Why not FP32? Ask Voltaire.

With the rise of AI, including hardware acceleration of AI, comes a renewed interested in efficient data types.  

In this spirit, we raise a toast to FP16 or float16, also known as half-precision floating point format, which can be a more appropriate format in some circumstances and some algorithms than what is known as single precision floating point which occupies 32 bits (and hence also called FP32 or float32).

But why use a less precise data type at all, when more precision options are available?

Half precision values are useful in applications where perfect precision is not required, such applications include image processing and neural networks.

FP16 is not to be confused with bfloat16 (Brain float16) which is a different format developed for Google Brain (now Google AI) with the explicit intent of accelerating machine learning and is used a variety of AI processors (including Google Cloud TPUs).