What is HPC ?

- What is performance?

Efficency utilization of computer hardware resources ,we want to improve on time to solution
we want to get the solution to the problem as fast as possible maybe we want to minimize the
energy to solution that also had performance computing and maybe high performance means
getting the best algorithms for doing the work what is the algorithm complexity there are
many ways we can define what high performance is .

HPC Modes:

-Resarch Codes

two modes of operation of HPC one is that we have research codes research code is that
someone doing the PhD thesis writing a code to run a simulation or to extract some data do
something with this thing and the only thing that matters is you need to get the code to run fast
to be able to solve our our scientific issues

-Production applications

the most typical production application is the weather climate forecasts where they have
parallel computer scenes since the 70s 80s. they need to run for a long time now long depends
on now when you talk about production the metric is not usually just go fast is usually time to
solution we need to get the weather forecast in three hours otherwise doesn't make any sense
to make the computation at all and then we see here that results are independent from the
hardware you don't care about which hardware you're running you need your solution by that
time so the user of this code doesn't care about hardware.

High Performance Code:

3x3 matrix multiplication I might have two different codes

since the operations here are actually commutative and associative I can take the loops and
change the order I can just change the order of the loops so a question that arises now is which
is faster between A and B.

B it's the fastest code because the the C++ layout memory of a multi-dimensional array .
assuming ethnicities are two-dimensional arrays then the layout memory of the two-
dimensional array is by rows so it means the first row is laid out for then the second row and
then the third row as we know the memory of the computer is a single file we put the first row
the second row third row etc
So A B and C will be laid out in in this way so when I'm accessing here for instance in the
innermost loop that the K index with will go to the A columns here and B rows here and it
means that in A I will go on the rows I will spend my column and that storage would ease
contiguous when I go on B in this guy here I'm going to jump from one row to the next row and
this is a problem because of caches. so when I load a value from RAM it will not just come a
value it will come a full line and this line will be put into the cache and then if I access the next
element in that line it will be much more efficient.

On the other side , I have swaped the j and k loops , now I will get the the C and B to be
accessed in row major so I have two arrays that are accessed in a cache friendly way and and A
will be next best..

HPC focus on hardware:

so C++ provides you facilities to access hardware information. computer has multiple cores they
run multiple threads at the same time and they access the data through the cache system now
if the thread access of value loads the cache line into the cache then if another thread has to
access a value which is close by to that one it's not the same value but one that is close by to
that one they will evict the cache line from one core and move it to another core this is called
coherency issue right we have something that eats cache lines moved back and forth and we
spend a lot of time doing that so for instance this hardware destructive interference size tells
us how far two variables should be so if you put the variable at the a two addresses that are too
close they will interfere in the cache so this number will tell you that this variable should be at
least this far otherwise you get destructive interference.

Constructive interference size is the opposite when we want this to happen. If we want to share
the data with cache lines what should I do , what is the size , how far should my variables be in
order to make this happen.

How many hardware threads can I run on my architecture

WHY C++ ?
In short, C++ is a highly portable language which offers zero-cost abstractions. Furthermore, we
believe that C++ provides programmers with the ability to write and manage large, expressive,
and robust code bases. Let's explore the meaning of each of these properties.

Zero-cost abstractions

We need abstractions such as functions, classes, data structures, layers and so on in order to
manage the complexity of a large-scale code base. But constantly adding abstractions and new
levels of indirection comes at a price — efficiency. This is where zero-cost abstractions plays its
role. A lot of the abstractions offered by C++ comes at a very low price. At a minimum, C++
offers efficient alternatives at hot spots where performance really is a concern.

With C++ you are free to talk about memory addresses and other computer related low-level
terms when needed.

Programming languages and machine code abstractions

In order to relieve programmers from dealing with computer-related terms, modern
programming languages use abstractions so that a list of strings, for example, can be handled
and thought of as a list of strings rather than a list of addresses that we may easily lose track of
if we make the slightest typo.

To give you some examples of C++ abstractions we will here show how a problem can be solved
in both C and C++.

Although the C++ version is still more of a robot language than a human language, a lot of
programming lingo is gone. Here are some of the noticeable differences between the preceding
two code snippets:

Abstractions in other languages

Most programming languages are based on abstractions, which are transformed into machine
code to be executed by the CPU. C++ has evolved into a highly expressive language just like
many of the other popular programming languages of today. What distinguishes C++ from most
other languages is that, while the other languages have implemented these abstractions at the
cost of runtime performance, C++ has always strived to implement its abstractions at zero cost
at runtime. This doesn't mean that an application written in C++ is by default faster than the
equivalent in, say, C#. Rather, it means that by using C++, you'll have explicit control of the
emitted machine code instructions and memory footprint if needed
Portability
C++ has been a popular and comprehensive language for a long time. It's highly compatible
with C and very little has been deprecated in the language, for better or worse. The history and
design of C++ has made it to a highly portable language, and the evolution of modern C++ has
ensured that it will stay that way for a long time to come. C++ is a living language and compiler
vendors are currently doing a remarkable job to implement new language features rapidly

Drawbacks of C++

The most severe of those shortcomings are long compilation times, the reliance on the manual handling
of forward declarations, header/source files, and the complexity of importing libraries.

There are, nonetheless, some shortcomings of C++, which are simply just shortcomings. The most severe
of those shortcomings are long compilation times, the reliance on the manual handling of forward
declarations, header/source files, and the complexity of importing libraries.

This is mainly a result of C++ relying on an outdated import system where imported headers are simply
pasted into whatever includes them

Another apparent drawback of C++ is the lack of provided libraries. While other languages usually come
with all the libraries needed for most applications, such as graphics, user interfaces, networking,
threading, resource handling, and so on

To summarize, although C++ has a steeper learning curve than most other languages, if used correctly,
the robustness of C++ is an advantage compared to many other languages. So, despite the outdated
import/library system of C++, we believe that C++ is a well suited language for large-scale projects, even
for projects where performance is not the highest priority

Summary
To summarize, although C++ has a steeper learning curve than most other languages, if used correctly,
the robustness of C++ is an advantage compared to many other languages. So, despite the outdated
import/library system of C++, we believe that C++ is a well suited language for large-scale projects, even
for projects where performance is not the highest priority.
Robustness
In addition to performance, expressiveness, and portability, C++ offers a set of language
features that gives the programmer the ability to write robust code. Robustness does not refer
to strength in the programming language itself – it's possible to write robust code in any
language. However, strict ownership of resources, const correctness, value semantics, type
safety, and deterministic destruction of objects are some of the features offered by C++ that
makes it easier to write robust code. That is, the ability to write functions, classes, and libraries
that are easy to use and hard to misuse.

C++ of today

To sum it up, C++ of today provides programmers the ability to write an expressive and robust
code base while still having the ability to target almost any hardware platform or real-time
requirements. Of the most commonly used languages today, C++ is the only one that gives all of
these properties

C++ compared with other languages

A multitude of application types, platforms, and programming languages have emerged since
C++ was first released. Still, C++ is a widely used language, and its compilers are available for
most platforms. The major exception, as of today, is the web platform, where JavaScript and its
related technologies are the foundation. However, the web platform is evolving into being able
to execute what was previously only possible in desktop applications, and in that context C++
has found its way into web applications using technologies such as Emscripten/asm.js and web
assembly

C++ vs Java

In order to understand how C++ achieves its performance compared to other programming
languages, I would like to discuss some fundamental differences between C++ and most other
modern programming languages.

Firstly, Java compile to bytecode, which is then compiled to machine code while the application
is executing, whereas C++ directly compiles the source code to machine code. Although
bytecode and just-in-time compilers may theoretically be able to achieve the same (or
theoretically, even better) performance than precompiled machine code, as of today, they
simply do not. To be fair though, they perform well enough for most cases
Secondly, Java handle dynamic memory in a completely different manner from C++. In Java,
memory is automatically deallocated by a garbage collector, whereas a C++ program handles
memory deallocations by itself. The garbage collector does prevent memory leaks, but at the
cost of performance and predictability.

Thirdly, Java places all its objects in separate heap allocations, whereas C++ allows the
programmer to place objects both on the stack and on the heap. In C++ it's also possible to
create multiple objects in one single heap allocation. This can be a huge performance gain for
two reasons: objects can be created without always allocating dynamic memory, and multiple
related objects can be placed adjacent to one another in memory.

Comparing C++ with JAVA

Take a look at how memory is allocated in the following example. The C++ function uses the
stack for both objects and integers; Java places the objects on the heap:

Now take a look at the next example and see how an array of Car objects are placed in memory
when using C++ and Java respectively:…

The C++ vector contains the actual Car objects placed in one contiguous memory block,
whereas the equivalent in Java is a contiguous memory block of references to Car objects. In
Java, the objects has been allocated separately, which means that they can be located
anywhere in the heap. This affects the performance as Java has to execute seven allocations
instead of one. It also means that whenever the application iterates the list, there is a
performance win for C++, since accessing nearby memory locations is faster than accessing
several random spots in memory.