Category: query

Apache Flink


– Open source stream processor framework developed by the Apache Software Foundation (2016)
– Data streams with high data volume can be processed and analyzed with low delay and high speed

flink analytics
Flink provides various tools for efficient real-time processing of continuous data streams and batch data

Core functions

– diverse, specialized APIs:
→ DataStream API (Stream Processing)
→ ProcessFunctions (control of states and time; event states can be saved and timers can be added for future calculations)
→ Table API
→ provides a rich set of connectors to various storage systems such as Kafka, Kinesis, Kubernetes, YARN, HDFS, Elasticsearch, and JDBC database systems

Stream Processing

pexels pixabay 2438
How to handle this flood of data?

== Data is processed continuously with a short delay
→ without intermediate storage of the data in separate databases
– several data streams can be processed in parallel
– Each stream can be used to derive own follow-up actions and analyses


Data can be processed as unbounded or bounded streams:

  • Unbounded stream

    • have a start but no defined end

    • must be continuously processed

  • Bounded stream

    • have a defined start and end

    • can be processed by ingesting all data before performing any computations(== batch processing)

– Flink automatically identifies the required resources based on the application’s configured parallelism and requests them from the resource manager.

In case of a failure, Flink replaces the failed container by requesting new resources.

– Stateful Flink applications are optimized for local state access

What is Microsoft’s .NET?


== collective term for several software platforms published by Microsoft (2000)
– for development and execution of application programs

Main components:

.NET Framework

.NET Core

Mono from Xamarin

overview dotNET

.NET Framework

is a run-time execution environment that manages apps that target

overview net frameworks


– Command line compiler (for Visual Basic.NET, C#, Jscript.NET)
– Software Development Kit (SDK) with tools, documentation and examples

Common Language Runtime (CLR)

– Runtime environment for the execution of a .NET application
– provides the just-in-time compiler
– provides numerous other basic services (garbage collector, exception handling, a security system and interoperability with non.NET applications)
– When a .NET application is started, Windows does not call the CLR itself directly, but first calls a so-called Runtime Host. This host loads the CLR and passes the entry point for the application to the CLR.

Runtime Hosts:
– Shell Runtime Host
– Internet Explorer Runtime Host

Language Integration

– enables the cooperation of different languages
→ Calling a program code written in another programming language
→ object-oriented languages can inherit classes written in another object-oriented language

Uniform class library

– NET Framework Class Library (FCL)
– extensive class library that can be used from all .NET languages
– implemented as a set of DLLs (Managed Code)

Component concept

– not only object-oriented, but also component-oriented
– At the center of the component concept are the so-called Assemblies (EXE, DLL)
== a composite of one or more MSIL files, where at least one of the files is a DLL or EXE/ non-MSIL files, so called resource files (database, graphic or sound files)
DLL assembly:
– a reusable software component that can be used by another assembly
EXE assembly:
– can be started as an independent application
– but can provide services for others

.NET Core

== universal open source development platform
– replaces the old .NET Framework (mixture of new implementation and redesign/refactoring of .NET Framework 4.x)
– faster than .NET Framework
– modularly constructed
– allows you to create .NET Core Apps, cross-platform, for Windows, macOS and Linux for x64, x86, ARM32 and ARM64 processors
→ different programming languages are supported (C#, F#, or Visual Basic)
→ Frameworks and APIs for the cloud, for IoT, for client user interfaces and for machine learning
– Modern language constructs like generics, Language Integrated Query (LINQ), and asynchronous programming

Language Integrated Query (LINQ)

== an expression that retrieves data from a data source
– are usually written in a specialized query language
– a new query language for each type of data source or data format they need to support
→ LINQ provides a consistent model for working with data in different types of data sources and formats
– Queries are always object based

LINQ query operations consist of three actions:
– Retrieving the data source
– Creating the query
– Executing the query

overview linq


== alternative, open source implementation of Microsoft’s .NET Framework
→ Development of a platform independent software based on the standards of the Common Language Infrastructure, ECMA, and the programming language C# shall be enabled

Additional functions for .Net

– Interfaces for operating system related functions under Unix
– Comprehensive Technology Coverage
– Binary Compatible
– Multi-Language(VB 8, Java, Python, Ruby, Eiffel, F#, Oxygene …)
– Changes can be made to already compiled code
– easy generation of native code


– C# Compiler

– Mono Runtime

– .NET Framework Class Library

– Mono Class Library

.NET 5.0

→ In autumn 2020 all three platforms (.NET Core, .NET Framework, Mono) will be merged

overview net5