Time Synchronization in Distributed Systems - A Different Take

Distributed systems employ a lot of synchronization - data synchronization (get all nodes consistent on data) as well as time synchronization (so every node has the same view of time).

Time synchronization can be achieved via protocols like NTP and PTP (the latter is precision time protocol - for applications like high frequency trading), but also logical systems like Lamport timestamps which order events without relying on physical time.

Lamport timestamps are a logical clock mechanism to capture chronological and causal relationships in a distributed system. Leslie Lamport introduce this as far back as 1978.

A variation of logical clock known as a vector clock is used in a variety of distributed database systems.

Relationship of WMIC to WBEM

WMI is the Microsoft implementation of Web-based Enterprise Management (WBEM), a set of systems management technologies for use in distributed systems.  It is based on common standards like the Common Information Model (CIM).

The WBEM initiative was started in 1996 by BMC Software, Cisco, Compaq, Intel and Microsoft.

The specification for the CIM is maintained by the DMTF (Distributed Management Task Force), an industry standards organization consisting of members and alliance partners collaborating on specifications. It also maintains a repository of its operating policies on issues including IP.

gRPC for .NET

The recommended library for using gRPC in .NET is grpc/grpc-dotnet under the Apache 2.0 license (The original implementation is now deprecated). 

gRPC is a (language agnostic) remote procedure call framework designed for high performance.

gRPC is often used with protobuf with coding structured around .proto files explained here.  One of the reasons for high performance is compact binary serialization of these so-called protocol buffers.

Common use cases include gRPC authentication (gRPC can converse with a variety of authentication systems), compression and other distributed communication scenarios. 

It's first commit was made internal to Google in December 2013 and was made public on Github in 2015. Google was doing RPC with protocol buffers for years and years,  but needed to expose APIs for public consumption. In 2017, gRPC was donated to the CNCF.  More complex features were incorporated like Flatbuffer, which is for serialization - where you can use the data without full deserialization (originally used for games). Observability features have also been built into gRPC since its initial release.

gRPC is used by Netflix and Facebook and Google of course.

Stateful Services in Service Fabric

Managing state reliably in a distributed system is a challenge. Service Fabric rises to the challenge. The key hazards of a distributed system:

1. Messages can get lost

2. Failure of a node

3. Communication channel stalls

So when a stateful service needs to persist state, it must use a Reliable Collection! SF provides reliable Dictionaries and reliable Queues to serve this end.

Reliable data structures are distributed data structures but can be used as if they are local data structures.

Reliable collections are persisted by a Reliable State Manager component (which is represented by a class in the Microsoft.ServiceFabric.Data namespace which lives in the Microsoft.ServiceFabric.Data.dll in the Azure SDK).

The states are replicated by a Transactional Replicator to secondary replicas for availability and reliability.

Most stateful services inherit from a StatefulService class.