Description

Span<T> is a new ref struct introduced in C# 7.2 specification. It is a stack-only type that allows memory operations without allocation so, used for instance in very large arrays, it can be a significant performance improvement.

It is only applicable if your code is based from .NET Core 2.1 and .NET Standard 2.1. There are tons of technical documentation available about Span<T>, this post is just going to be focused in a practical demo to compare the performance of the slice method. Span<T> can't work inside Anync methods but you can work around this issue easily creating a non Async local method.

Detailed information can be found in the official Microsoft link: https://docs.microsoft.com/en-us/dotnet/api/system.span-1?view=net-5.0

(...)
[Continue Reading]

Introduction

Weeks ago I decided to start creating an experimental home size "Big data" system based on Apache Spark. The first step for it is to create a distributed filesystem where Apache Spark will read and write eveything.

HDFS is the Hadoop distributed filesystem which provides features like: fault detection and recovery, huge datasets, hardware at data, etc... despite it is a Hadoop ecosystem piece, it works nice as the data distributed filesytem for Apache Spark.

(...)
[Continue Reading]