1
0

Update and new 2025 roadmap
All checks were successful
Build Jekyll weblog / build (push) Successful in 4s

This commit is contained in:
JackCarterSmith 2025-01-14 15:08:48 +01:00
parent 0bbfcbcd8e
commit a079606a1e
Signed by: JackCarterSmith
GPG Key ID: 832E52F4E23F8F24
10 changed files with 117 additions and 368 deletions

1
.gitignore vendored
View File

@ -3,4 +3,5 @@
.sass-cache/ .sass-cache/
_site _site
_old_posts _old_posts
_wip_posts
Gemfile.lock Gemfile.lock

View File

@ -40,6 +40,6 @@ permalink: /404.html
</div> </div>
<hr> <hr>
<div id="page-footer"> <div id="page-footer">
<p id="footer-text">Copyright &copy;2024 <a href="{{ "/about/" | prepend: site.baseurl }}">JackCarterSmith</a></p> <p id="footer-text">Copyright &copy;2025 <a href="{{ "/about/" | prepend: site.baseurl }}">JackCarterSmith</a></p>
</div> </div>
</body> </body>

View File

@ -77,7 +77,7 @@
<div class="footer-col footer-col-2"> <div class="footer-col footer-col-2">
<p>{{ site.description }}</p> <p>{{ site.description }}</p>
<br /> <br />
<p>Copyright &copy;2024 JackCarterSmith</p> <p>Copyright &copy;2025 JackCarterSmith</p>
</div> </div>
</div> </div>
</div> </div>

View File

@ -1,242 +0,0 @@
---
layout: post
title: Build a game engine from scratch
author: JackCarterSmith
categories: Programming C++ GameDev
thumbnail: build_an_engine
highlight: true
---
* Contents
{:toc}
Creating a game inevitably involves choosing an engine to support our assets and
mechanics. In 2023, the numbers of moderns game engines are quite large: Unity,
Unreal, Godot, Cry, Source, idTech, OGRE, Monogame etc.
Each one came with it's own features, and also it's own **weakness**. When I talk about
benefits or consequences of an engine implementation, can be anything from what you
can do with it to design rules.
When I started to look for an engine for my game rewrite idea, there are two questions
that came up to me:
- should the engine adapt to the game, or the game to the engine?
- and why not design an engine?
To these questions, I've found many opinions and explainations, not a really precise answer.
So after some comparisons and a pinch of personal opinion, I can now answer an "engine selection"
checklist:
1. You have an idea of your game concept, artistic touch, mechanics and features.
2. Search for an existing engine that suits your game's needs. Check not only the advantages, but also the constraints.
3. If there is no matching engine, can you adjust your game's specifications to select a candidate?
4. If the previous answer is always no, then you should consider making your own game engine.
Just as your game exploits its engine, the engine exploits your skills to produce a
game that more or less *run* So don't underestimate the importance of **working your coding-skills**.
The advantage of having so many game engines today is that you can find engines to
suit all levels! If you have already practiced programming, [Unity][link_unity] or
[Godot][link_godot] can do the job for you. And if you didn't, [GameEditor][link_gameeditor]
could be an option, but keep in mind that it's your programming skills that open up
game design possibilities.
You can also choose **to design your own engine** just for the *state-of-the-art* aspect
or as a personal technical challenge. It's not a bad choice, and it's the job of
this series of articles to explain why I chose this option.
> Be **patient**, really! Don't give up on setbacks.
### Why use your own game engine?
Freedom! Since you define the **design rules** using your own engine implementation, you can
create many more aspects than texture or shaders, such as physics or rendering methods.
This is the major point in favor to make your own engine. There are several points
that follow more or less directly from this:
- better game specific optimizations,
- mastering/how-its-work low-level aspects (fast debugging),
- no time spend reading engine documentation to find out how to implement our feature,
- no generic features overhead you didn't need in your game (like multiplayer protocol on singleplayer game),
- no engine licensing constraints (others than the lib you used inside of it).
From my personnal opinion, the biggest flaw I see in existing engines, which is ironically
their strength, is their *versatility*! In order to enable a wide range of uses for their engine,
these come with an abstraction layer that not only makes the engine (and the game as a result)
heavy in terms of disk size, but also in terms of performance. Some allow you to modify the
rendering engine to a certain limit, in order to adapt to the needs of devs, but this requires
work that is not negligible when compared to the dev time of an engine. And when it comes to
file size, you have to bear in mind that the size of the Unreal Engine (UE), compiled on its own,
is close to **a gigabyte of disk space**...
Indirectly, you'll learn more than any *engine users* as you need to develop others
aspects you may not have thought of before like geometric computation and physics
algorithms. As you build your own implementation of physics and rendering pipeline,
you feel more confident about what your game is going on, you know what's going
on behind the scenes! Yes, you'll take time to develop (again) all the basics elements,
but it's an advantage you'll keep throughout your development for this and future projects.
> Like athletes, warming up with small exercises helps you keep up the pace during the real effort.
Yeah but... It's not all benefits either. And these are all good reasons that should be taken
into account when choosing between a *home-made* engine and an existing one.
Firstly, the biggest problem is the time. Making a game engine from scratch require
a lot, a loooooot of time before displaying your first image on screen (as you'll see in
the following articles, it took me almost **one year to display something** that suited me!) and an
another waiting after that to make it playable through an unoptimized release! So... Another advice:
be patient, really! Don't give up in the face of setbacks, because yes, you'll have them,
and it's even important to have them in order to understand and better manage what comes next.
You've an impressive numbers of features to implement in order to make just the game
rendering something on screen. And the less comfortable you are with the language, the
operation of a GPU and its drivers, memory management, 3D on a 2D surface... The more
you'll have to retrace your steps to code these interfaces rather than the game itself.
I think, this is why creating an engine is quite *complex*.
> All vast subjects like 3D engine require discipline and rigor to understand and to master it.
### Engine development choices
For any engine, you need to interface with your GPU. Long time ago, every GPU vendors implements
its own interface to allow user to make thing on screen. For each GPU in the worlds you need to
have a portion of code to implement it! That's really annoying... And that's why, a few years later,
OpenGL and then Microsoft DirectX, have set up an API to standardize everything and every GPU since
feature an interface level for OpenGL, Vulkan and DirectX. All the low-level hardware interface with
GPU is done through these APIs, like memory transfer at chip-level and others DMA access.
And for my first engine, I have choosen as API... DirectX... **DX-12** yup.
I can't say today if it's a good or very bad choice, I've read everywhere about starting with
OpenGL as a first engine for a lot of good reasons:
- easy to handle
- lot of documentations
- portable between OSs
Because yes, DirectX is **proprietary to Microsoft** and can be used only on Windows environnement!
And as if that weren't enough: the version 12 is considered to be the most complicated to apprehend
because of the redesign of the graphic pipeline to take full advantage of a multi-threaded environment.
And why not older *simple* version like DX9? Even if I learned DX9, I'd to spend more time unlearning
a lot to adapt to DX12. Bearing in mind that, with the exception of DX11 and 12, the other versions
are considered obsolete for a new design. Although this goes against what I was saying about the
importance of doing small exercises before the big effort, I can explain it.
I really want to focus on the mechanics of a game engine, and how it do the things. DX12 leaves memory
management to the developer and you need to understand how the resources are shared between GPU,
CPU and RAM. Since I'm primarily interested in designing a *state-of-the-art*, technically challenging
engine, using DX12 ask more times and efforts but it's worth it! I get a better management of what the
GPU and CPU does rather than letting the driver "do its thing". And it's always possible to add a
handler to manage both DirectX and OpenGL in function of the running OS or the user's choice. We'll
talk about that in a future post.
No matter the API you choose, you need one more important things: a **good programming language**! And
no, Python is **not**. Making a game in Python may be fun, but it's not really technically interesting,
it's not designed for that. Python will slow your game, your dev and it's not adapted to debug properly
DX or GL interface. Java can do the job, like Minecraft using OpenGL, but it's a *virtual-machine language*.
The optimization level is a pain in ass. C# can have it's chance, used by MonoGame, the performance are
greats and compatible with DX (also know as SharpDX). But C# is a proprietary language of Microsoft.
If I choose C#, I'll get more troubles to debug my code and I definetly close the door for porting code to other OSs.
And C++? A lot of books about DX use this language, it's cross-platform, always documentated and in
development, multi-threads ready, objects oriented and have a lot of existing usefull library like [Boost][link_boostlib].
In addition a lot of examples of games codes are made in C++.
Here's a list of the others tools I used during my development, which I will detail if necessary:
- Visual Studio 2022 - IDE with graphics oriented tools
- [CMake - Build manager][link_cmake]
- [Conan - Dependencies manager][link_conan]
- [Git - File version control][link_git]
- [PIX - GPU rendering debugger and analyzer][link_pix]
- [RenderDoc - Another GPU rendering debugger][link_renderdoc]
Let's go for a ~~sadomasochistic~~ adventure with C++ and DX12. What could go wrong?!
**Now let's talk less and code more! See you in next post ==>**
### Some books to begin
Rather than list books like that without much explanation, I'll mention which books I've read
and what they've brought me.
The first book I've read and revisit most often is [3D Game Programming with DX12 by F.D.Luna][link_gameprogdx12].
This book cover all the mains topics about DX12 usage with some examples. The only downside,
is that it's not easy to know exactly what functions all the elements indicated as structures
or methods perform. I needed to read again several times and experiment directly in a program to
understand the use of a feature. But beside of that, the book is very comprehensive for
running a rudimentary engine, with simple lighting, shadows and textures.
I've got the DX9 and DX11 editions of this book for comparison and... Same topics, just adapted
for DX11 and DX9 pipeline. DX9 has far fewer subjects because of its lack of shaders prior to
9.0c version.
The second book is not really related to game engine, because it's the [Holy Bible of C++11 by B.Stroustrup][link_holycpp]!
A complete dictionary of all the C++ features and keywords, really usefull when I need to fully
exploit a function from this language. Or maybe choose an another option to my issue than the one
I thought earlier. *B.Stroustrup* said that C++ was a language that had to respond to a problem
with a language feature!
I plan to buy the new edition of C++17 soon to benefit from interesting multithreading features.
I also read two other books: [Game Engine Architecture 3rd Ed. by J.Gregory][link_gameenginearch] and
[Game Programming Patterns by R.Mystron][link_gameprogpatterns]. Both doesn't explictly show **what to code**,
but **how to code** a good game engine. The first book describe each parts of a "modern" game engine,
like physics, audio, user inputs, files management, graphics (with DX in my case), etc.
The second one talk about the abstracts concepts of game engine like object memory management,
pointers good practice, game loop, and others importants things that we don't necessarily think about,
but which are necessary for the proper operation of a game engine.
On top of that, you have the [online DX12 documentation][link_d3d12doc] which will go into greater
detail than F.D.Luna's book. Same advice for the Windows API to manage the window and the system messages.
All these books have helped me a great deal in clarifying aspects of a game engine that I couldn't see
at first. But there's **no one right way to design a game engine**. Unlike a calculator program, where the
sole aim is to perform calculations as efficiently as possible, our engine must meet several needs in
terms of functionality to match the game we want to make. Most books and examples only give us one way
of doing things, although sometimes there's no other way than to go through this or that method to achieve
a task, the rest is up to our own judgement.
> I really like the low-level programming because it's like a puzzle-game to make a game.
My last piece of advice is to **take inspiration from existing engines**. And if the source code is available,
try to understand **why it's implemented the way it is**. Others previous developers have run into the same problems
you're about to encounter. For the time being, it would be a waste of time (and possibly of motivation)
to repeatedly get *our feet stuck in the carpet*. Above all, the aim of this blog is
to share successes and mistakes, that's what knowledge is all about.
### It's (not) a trap!
> This is a recurent chapter in a large part of my topics to describe, from my own experience
> and point of view, the points on which I may have found myself stuck, or which at some point
> required me to pause and reflect.
>
> Use them as you like!
When I choose to use DX12 as my first API for a 3D engine, the price to pay for that is to take
even more time to complete the first draw on screen. That's part of the reason why it took me a
year to achieve this result. Today I'm on the 4th attempt/iteration, and that's not counting tests
on test code apart from the engine sources. I'll give you all details in upcoming posts!
So why not using Vulkan if I wanted a challenge? Some reasons for that: **Vulkan isn't so well documented**
than DirectX, or at least it was easier to find DirectX docs and books. And it's for me a good
sign to avoid or to limit the *demotivation effect*: the more time and energy you invest in
something that doesn't bring you any satisfaction within the limits of your patience, the more
you give up or *make a break* on the project. The greater the challenge, the greater the risk
of *breaking*. This is normal, and it's through this kind of failure that we learn to know our
limits. So I keep Vulkan as a next feature or evolution for now.
[link_gameeditor]: http://game-editor.com/static/Main_Page.html
[link_unity]: https://unity.com/fr
[link_godot]: https://godotengine.org/
[link_boostlib]: https://www.boost.org/
[link_cmake]: https://cmake.org/
[link_conan]: https://conan.io/
[link_git]: https://git-scm.com/
[link_pix]: https://devblogs.microsoft.com/pix/download/
[link_renderdoc]: https://renderdoc.org/
[link_gameprogdx12]: https://amzn.eu/d/9aTUE79
[link_holycpp]: https://amzn.eu/d/cN7pZ7Q
[link_gameenginearch]: https://amzn.eu/d/bEcBNXE
[link_gameprogpatterns]: https://gameprogrammingpatterns.com/
[link_d3d12doc]: https://learn.microsoft.com/en-us/windows/win32/direct3d12/directx-12-programming-guide

View File

@ -1,124 +0,0 @@
---
layout: post
title: Setup for a game engine development
author: JackCarterSmith
categories: Programming C++ GameDev
thumbnail: engine_tools_setup
highlight: true
---
* Contents
{:toc}
My first point of reflection was to choose/find the tools I needed to develop my game engine.
As I intend to use Direct3D for GPU interface and rendering, my developement platform will be
*Windows* obligatory (but maybe not, see below...)
For those who haven't followed the reasons behind my choice of API and platform, I invite them
to have a look at my [previous blog post]({% post_url 2023-09-10-build-a-game-engine %}).
I'm used to [Eclipse IDE][link_eclipse] for C++ developement with [CMake][link_cmake] as project manager. But
as Direct3D has been part of the Windows SDK since version 10, support for it will be more complete
if I use [VisualStudio and the MSVC toolchain][link_vs]. Sometimes, for a quick modification or
comparison, I use VisualStudio Code.
It may seem superfluous, but I always recommend a large whiteboard to put your ideas on. It's
quite effective for me in clearing up the mess in my head.
### Direct3D and DirectX
Throughout this blog, you'll come across the terms DirectX, Direct3D, DXGI and so on. A brief
aside to explain exactly what I'm talking about is in order to avoid losing anyone from the outset.
DirectX was developed from version 1 to 9 from Windows 95 to Windows XP to make graphics game
development more accessible (before DirectX, a *heavier* version of OpenGL existed, but could
only be used on specialized machines). At the time, the real advantage of DirectX was its ability
to interface with the graphical display as well as the *human-machine interface* (keyboard, joystick, etc.)
or sound. Using DirectX means you only need one library to manage user interfaces!
As you can see, DirectX is more than just a graphics API... In fact, DirectX is a set of multiple
libraries for gaming:
- **Direct3D/2D** for the graphic rendering,
- **Xaudio** (aka. DirectSound) for... sound,
- **DirectPlay** for multiplayer session,
- **Xinput** (aka DirectInput) for Human Interface Device (HID),
- **DXGI** for hardware management like GPU card selection,
- And others for specialized usage.
At this time, DirectX is used for a mono-thread usage and all action taken inside user code, are
executed immediatly (no command list). Multi-threading will not be available until version 11 on
Windows Vista.
Versions 11 and 12 essentially change the way resources are allocated in video memory and how GPU
instructions are processed by it.
Since May 20, 2020, DirectX12 has been ported to linux as open source. This is particularly
interesting for the idea of porting my engine to other platforms. *I wasn't aware of this when I
chose this API, but it makes me all confident for the future.*
DirectX is included as standard in the Windows SDK, which is installed with Visual Studio (as long
as you select **Game Development with C++** in the installation options). So let's move on to the next part...
### VisualStudio setup
I'm using *VisualStudio Community edition* for my game engine developement. It has all the tools you
need for design and debugging. Although it can be heavy to install, I can recommend to the purist
to install only the SDK with the MSVC toolchain, but I won't detail the procedure here (far too
long and technical for this post).
For the setup options, the minimal is:
- **Game Development with C++**,
- Windows SDK (normally included by default),
- HLSL tools,
- Graphics tools (located in Windows *Optionals Features* in Apps settings).
### Others usefull graphic debugging tools
In addition to code design and debugging tools such as Visual Studio, I use graphical debugging
tools that are specifically designed to check the graphical pipeline.
I recommend two of them, mainly because I'm used to using them, but they were the ones who got
me up quickly:
- **[PIX][link_pix]** is a precise measurement and tunning tools of render timing. Developed by
Microsoft, it come as a standalone program (for GUI) and a library to integrate into our code.
When our game start, the PIX program will connect to the [lib interface][link_pixlib] and begin to collect datas
about the current running graphic pipeline, getting all GPU resources and event timing on the move.
It can also be launch on a separate machine from the one where the game is run and connect to it
through network. It only work with Direct3D API.
- **[RenderDoc][link_renderdoc]** is equivalent to PIX in terms of features but it's open source
and can have plugins and scripts to extends its limits. Unlike PIX, it support Direct3D, OpenGL
and Vulkan API. It plug on app using debug layer of the API (who need to be integrated by dev
in its code). Linux support is a plus!
Both of this tools work better on Direct3D-12 API, with retro-compatibility to version 11.
### Project template and others examples
My main aim is to give you my experience of game engine development. I won't go into detail here
on how to create a new project in VisualStudio.
With the DirectX suite integrated into the IDE, you'll have access to 4 template variants:
- Direct3D-11 with or without Device Resource (DR),
- Direct3D-12 with or without DR.
DR handle the *boiler-plates* of Direct3D and DXGI instance. And it give some example of how basically
interface with your GPU card and render an *empty* scene on your screen. I recommand using this in first
time and after some tries, you can make your own implementation. Try, experiment, fail... That the **only**
good method to learn effectively!
But if you're feeling a bit lost on VisualStudio, I'd advise you to go [here][link_msdx12quick] or [there][link_msdx12flow].
[link_eclipse]: https://www.eclipse.org/
[link_vs]: https://visualstudio.microsoft.com/
[link_cmake]: https://cmake.org/
[link_pix]: https://devblogs.microsoft.com/pix/download/
[link_pixlib]: https://www.nuget.org/packages/WinPixEventRuntime
[link_renderdoc]: https://github.com/baldurk/renderdoc
[link_msdx12quick]: https://learn.microsoft.com/en-us/windows/win32/direct3dgetstarted/building-your-first-directx-app
[link_msdx12flow]: https://learn.microsoft.com/en-us/windows/win32/direct3d12/creating-a-basic-direct3d-12-component

View File

@ -0,0 +1,13 @@
---
layout: post
title: CPU Vectorized Acceleration (SIMD)
author: JackCarterSmith
categories: Programming C C++
thumbnail: cpu_simd
highlight: true
---
* Contents
{:toc}
WIP

View File

@ -0,0 +1,101 @@
---
layout: post
title: Journey into 3D engine design - P1.Introduction
author: JackCarterSmith
categories: Programming C++ GameDev
thumbnail: build_an_engine
highlight: true
---
* Contents
{:toc}
For many years now, the performance of computers has been constantly evolving: the number of transistors
has increased transistors, reduced size, increased speed, reduced power consumption...
All these improvements have enabled computers to carry out more and more complex calculations in large
calculations. One of its applications is the display of a virtual 3D environment on a physical 2D screen
(your computer screen)! 3D rendering engines are now used everywhere: in entertainment but also in
architecture, aeronautics and the automotive industry.
Nowadays, it's very complicated to define what a 3D engine should be able to do. is accompanied by extensions,
generally related to the use of the software, but which end up hiding the the "basis" of 3D rendering on
a computer.
In this "*Journey into 3D engine design*" series, I'm going to look at the basics and write a pared-down
engine, retaining only the minimum that's useful. This base can later be used to understand how graphics
APIs work (Vulkan, Direct3D, Metal, etc.) and to create portions that can be reused in embedded systems
that don't have hardware acceleration (GPU).
I haven't planned any particular guidelines for these blog posts. I'll be writing the various topics
progress and the various issues I come across!
![SFML exemples]({{ "/static/images/posts/computer_nerd.jpg" | prepend: site.baseurl }} "Trust me, I'm an engineer!")
### __My problem with Unreal, Unity, Godot, etc.__
The WWW is full of "how to make a 3D game" tutorials. In fact, there's no shortage of these videos and blogs...
However, there are 2 types of tutorial: those that explain how to use an engine and create a game using it, and
those that explain how to create a game using graphics APIs such as OpenGL directly. There's actually a third,
less ordinary category (and here we're curious about the unusual!), which really starts from 0, no APIs or very
few libraries (apart from stdlib), including 3D maths libraries. This might seem like reinventing the wheel, but
you have to ask yourself: is my aim to study the wheel? Then yes, reinventing the wheel is essential.
So yes, in this series, no explanation of "how to make AAA with UE5" but rather details on how and **why pixels**
**are displayed on our screen simply from NUMBERS!** Do you think I'm making a fuss? It's up to you, but we're going
to eat maths, a lot! Guaranteed.
For the start, I'm going to use [SFML][link_sfml] (Simple and Fast Multimedia Library) for simplified management
of the OS window, without messing up the rather *OS-management-oriented* function code too much. I plan to remove
it later in the series, for a possible port to embedded. In addition, in order to have a practical example of
my post on [CPU optimisations]({% post_url 2024-03-01-cpu-vectorized-acceleration %}), I'll be adding **SIMD**
**instructions** equivalents alongside the implementation. This 3D engine will in fact be CPU-only in order to ensure
that it is as portable as possible on architectures that don't have a dedicated GPU or compatibility with standard
graphics APIs (although this is rare when a GPU is present).
Some engines are in the form of a framework or library and need to be used by a master application to tell it what to do,
while others combine the engine directly into the application, sacrificing the engine's modularity but making the
code easier to read for our purposes.
In conclusion, there are no good or bad reasons for creating your own engine, whether to achieve something very
specific artistically, or to get the best performance for a given gameplay. Obviously, this involves going down
the "long path" and often spending several hours on basic notions that are essential to the concept of a graphics
engine in general.
No pain, no gain.
![SFML exemples]({{ "/static/images/posts/sfml_games.jpg" | prepend: site.baseurl }} "A lot of indies games have been develop using SFML")
### __Minimum equipment__
SFML offers several function groups for graphics, audio and networking. For the moment, I will only need
the graphics part, in order to manage the system window and the rendering display. SDL was also a
potential candidate for this use, but for this project I wanted to make up my mind about SFML.
Apart from SFML, the mathematical part will be supported by a slightly modified and reduced implementation of [Direct3DMath][link_d3dmath].
The SIMD acceleration it offers is not negligible for CPU-only.
Multi-threading is also envisaged through the use of the **pthread** lib. More versatile than std::thread from
libstdc++ and less obscure than OpenMP in use (we'll come back to this).
Since we're recreating a graphical flow from scratch, we'll need to create our own tools for debugging rendering
and performance measurement tools.
I often work with the GNU toolchain (version 14 at the time of writing) and with the [Conan][link_conan] package manager.
The project itself is configured with [CMake][link_cmake]. From time to time I check compatibility with Windows
via the MSVC compiler. **MinGW** can also be used without contraindication.
And we'll try to keep to this strict minimum in order to lighten the code by keeping only what is necessary
for on-screen rendering, and also to improve the portability of the programme to more critical platforms
in terms of storage and performance.
See you in the next post!
**[END_OF_REPORT-250114.1459]**
[link_cmake]: https://cmake.org/
[link_conan]: https://conan.io/
[link_d3dmath]: https://github.com/microsoft/DirectXMath
[link_sfml]: https://www.sfml-dev.org/

Binary file not shown.

After

Width:  |  Height:  |  Size: 204 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB