Scala vs Java - The Duel (Both JVM languages)

Scala and Java are highly compatible as both are designed to run beautifully on the JVM (the JVM spec should be read/re-read periodically for all serious Java enthusiasts).

Scala uses lambda expressions more aggressively than Java. Both use invokedynamic under the hood.

Scala dominates now in distributed systems (Spark, Akka) particularly for data engineering and ML pipelines, as well as high performance back-ends.

The following is a good website to keep up to speed on Java: Dev.java: The Destination for Java Developers.

Learning Spark from the Latest Docs

Spark is best learned from the various programming guides. 

Access to these can be found from links on its latest documentation page.

These include a Quick Start, an RDD guide, an overview of Datasets and DataFrames and at least two guides on Streaming (1) Structured Streaming, and (2) the legacy functionality of Spark Streaming. 

MLib is also worth understanding, as is GraphX for graph processing, including graph parallel processing.

In terms of interfacing with other languages, Spark and Python is encouraged, whereas Spark and R is deprecated (through PySpark and SparkR respectively).

You should also be familiar with Spark Declarative Pipelines (SDP).

There is an associated knowledge base around deployment - including making Spark play well with Hadoop, Kubernetes and Amazon Elastic Compute (EC2).

Big Data's New Vacation Home - The Lakehouse; Microsoft's Approach

The lakehouse concept combines the capabilities of data lakes (which have scalability qualities) and data warehouses (which have advanced query functionality).

Microsoft Fabric's resources on Data Engineering delves into the concept of data lakehouse (and Microsoft's SaaS implementation, OneLake, billed as OneDrive for data)  with notes on how the lakehouse makes use of Apache Spark.