diff --git a/_posts/2023-09-10-build-a-game-engine.markdown b/_posts/2023-09-10-build-a-game-engine.markdown new file mode 100644 index 0000000..da54eec --- /dev/null +++ b/_posts/2023-09-10-build-a-game-engine.markdown @@ -0,0 +1,242 @@ +--- +layout: post +title: Build a game engine from scratch +author: JackCarterSmith +categories: Programming C++ GameDev +thumbnail: build_an_engine +highlight: true +--- + +* Contents +{:toc} + +Creating a game inevitably involves choosing an engine to support our assets and +mechanics. In 2023, the numbers of moderns game engines are quite large: Unity, +Unreal, Godot, Cry, Source, idTech, OGRE, Monogame etc. +Each one came with it's own features, and also it's own **weakness**. When I talk about +benefits or consequences of an engine implementation, can be anything from what you +can do with it to design rules. + +When I started to look for an engine for my game rewrite idea, there are two questions +that came up to me: + +- should the engine adapt to the game, or the game to the engine? +- and why not design an engine? + +To these questions, I've found many opinions and explainations, not a really precise answer. +So after some comparisons and a pinch of personal opinion, I can now answer an "engine selection" +checklist: + +1. You have an idea of your game concept, artistic touch, mechanics and features. +2. Search for an existing engine that suits your game's needs. Check not only the advantages, but also the constraints. +3. If there is no matching engine, can you adjust your game's specifications to select a candidate? +4. If the previous answer is always no, then you should consider making your own game engine. + +Just as your game exploits its engine, the engine exploits your skills to produce a +game that more or less *run* So don't underestimate the importance of **working your coding-skills**. +The advantage of having so many game engines today is that you can find engines to +suit all levels! If you have already practiced programming, [Unity][link_unity] or +[Godot][link_godot] can do the job for you. And if you didn't, [GameEditor][link_gameeditor] +could be an option, but keep in mind that it's your programming skills that open up +game design possibilities. + +You can also choose **to design your own engine** just for the *state-of-the-art* aspect +or as a personal technical challenge. It's not a bad choice, and it's the job of +this series of articles to explain why I chose this option. + +> Be **patient**, really! Don't give up on setbacks. + +### Why use your own game engine? + +Freedom! Since you define the **design rules** using your own engine implementation, you can +create many more aspects than texture or shaders, such as physics or rendering methods. +This is the major point in favor to make your own engine. There are several points +that follow more or less directly from this: + +- better game specific optimizations, +- mastering/how-its-work low-level aspects (fast debugging), +- no time spend reading engine documentation to find out how to implement our feature, +- no generic features overhead you didn't need in your game (like multiplayer protocol on singleplayer game), +- no engine licensing constraints (others than the lib you used inside of it). + +From my personnal opinion, the biggest flaw I see in existing engines, which is ironically +their strength, is their *versatility*! In order to enable a wide range of uses for their engine, +these come with an abstraction layer that not only makes the engine (and the game as a result) +heavy in terms of disk size, but also in terms of performance. Some allow you to modify the +rendering engine to a certain limit, in order to adapt to the needs of devs, but this requires +work that is not negligible when compared to the dev time of an engine. And when it comes to +file size, you have to bear in mind that the size of the Unreal Engine (UE), compiled on its own, +is close to **a gigabyte of disk space**... + +Indirectly, you'll learn more than any *engine users* as you need to develop others +aspects you may not have thought of before like geometric computation and physics +algorithms. As you build your own implementation of physics and rendering pipeline, +you feel more confident about what your game is going on, you know what's going +on behind the scenes! Yes, you'll take time to develop (again) all the basics elements, +but it's an advantage you'll keep throughout your development for this and future projects. + +> Like athletes, warming up with small exercises helps you keep up the pace during the real effort. + +Yeah but... It's not all benefits either. And these are all good reasons that should be taken +into account when choosing between a *home-made* engine and an existing one. + +Firstly, the biggest problem is the time. Making a game engine from scratch require +a lot, a loooooot of time before displaying your first image on screen (as you'll see in +the following articles, it took me almost **one year to display something** that suited me!) and an +another waiting after that to make it playable through an unoptimized release! So... Another advice: +be patient, really! Don't give up in the face of setbacks, because yes, you'll have them, +and it's even important to have them in order to understand and better manage what comes next. + +You've an impressive numbers of features to implement in order to make just the game +rendering something on screen. And the less comfortable you are with the language, the +operation of a GPU and its drivers, memory management, 3D on a 2D surface... The more +you'll have to retrace your steps to code these interfaces rather than the game itself. +I think, this is why creating an engine is quite *complex*. + +> All vast subjects like 3D engine require discipline and rigor to understand and to master it. + +### Engine development choices + +For any engine, you need to interface with your GPU. Long time ago, every GPU vendors implements +its own interface to allow user to make thing on screen. For each GPU in the worlds you need to +have a portion of code to implement it! That's really annoying... And that's why, a few years later, +OpenGL and then Microsoft DirectX, have set up an API to standardize everything and every GPU since +feature an interface level for OpenGL, Vulkan and DirectX. All the low-level hardware interface with +GPU is done through these APIs, like memory transfer at chip-level and others DMA access. + +And for my first engine, I have choosen as API... DirectX... **DX-12** yup. + +I can't say today if it's a good or very bad choice, I've read everywhere about starting with +OpenGL as a first engine for a lot of good reasons: + +- easy to handle +- lot of documentations +- portable between OSs + +Because yes, DirectX is **proprietary to Microsoft** and can be used only on Windows environnement! +And as if that weren't enough: the version 12 is considered to be the most complicated to apprehend +because of the redesign of the graphic pipeline to take full advantage of a multi-threaded environment. +And why not older *simple* version like DX9? Even if I learned DX9, I'd to spend more time unlearning +a lot to adapt to DX12. Bearing in mind that, with the exception of DX11 and 12, the other versions +are considered obsolete for a new design. Although this goes against what I was saying about the +importance of doing small exercises before the big effort, I can explain it. + +I really want to focus on the mechanics of a game engine, and how it do the things. DX12 leaves memory +management to the developer and you need to understand how the resources are shared between GPU, +CPU and RAM. Since I'm primarily interested in designing a *state-of-the-art*, technically challenging +engine, using DX12 ask more times and efforts but it's worth it! I get a better management of what the +GPU and CPU does rather than letting the driver "do its thing". And it's always possible to add a +handler to manage both DirectX and OpenGL in function of the running OS or the user's choice. We'll +talk about that in a future post. + +No matter the API you choose, you need one more important things: a **good programming language**! And +no, Python is **not**. Making a game in Python may be fun, but it's not really technically interesting, +it's not designed for that. Python will slow your game, your dev and it's not adapted to debug properly +DX or GL interface. Java can do the job, like Minecraft using OpenGL, but it's a *virtual-machine language*. +The optimization level is a pain in ass. C# can have it's chance, used by MonoGame, the performance are +greats and compatible with DX (also know as SharpDX). But C# is a proprietary language of Microsoft. +If I choose C#, I'll get more troubles to debug my code and I definetly close the door for porting code to other OSs. +And C++? A lot of books about DX use this language, it's cross-platform, always documentated and in +development, multi-threads ready, objects oriented and have a lot of existing usefull library like [Boost][link_boostlib]. +In addition a lot of examples of games codes are made in C++. + +Here's a list of the others tools I used during my development, which I will detail if necessary: + +- Visual Studio 2022 - IDE with graphics oriented tools +- [CMake - Build manager][link_cmake] +- [Conan - Dependencies manager][link_conan] +- [Git - File version control][link_git] +- [PIX - GPU rendering debugger and analyzer][link_pix] +- [RenderDoc - Another GPU rendering debugger][link_renderdoc] + +Let's go for a ~~sadomasochistic~~ adventure with C++ and DX12. What could go wrong?! + +**Now let's talk less and code more! See you in next post ==>** + +### Some books to begin + +Rather than list books like that without much explanation, I'll mention which books I've read +and what they've brought me. + +The first book I've read and revisit most often is [3D Game Programming with DX12 by F.D.Luna][link_gameprogdx12]. +This book cover all the mains topics about DX12 usage with some examples. The only downside, +is that it's not easy to know exactly what functions all the elements indicated as structures +or methods perform. I needed to read again several times and experiment directly in a program to +understand the use of a feature. But beside of that, the book is very comprehensive for +running a rudimentary engine, with simple lighting, shadows and textures. + +I've got the DX9 and DX11 editions of this book for comparison and... Same topics, just adapted +for DX11 and DX9 pipeline. DX9 has far fewer subjects because of its lack of shaders prior to +9.0c version. + +The second book is not really related to game engine, because it's the [Holy Bible of C++11 by B.Stroustrup][link_holycpp]! +A complete dictionary of all the C++ features and keywords, really usefull when I need to fully +exploit a function from this language. Or maybe choose an another option to my issue than the one +I thought earlier. *B.Stroustrup* said that C++ was a language that had to respond to a problem +with a language feature! + +I plan to buy the new edition of C++17 soon to benefit from interesting multithreading features. + +I also read two other books: [Game Engine Architecture 3rd Ed. by J.Gregory][link_gameenginearch] and +[Game Programming Patterns by R.Mystron][link_gameprogpatterns]. Both doesn't explictly show **what to code**, +but **how to code** a good game engine. The first book describe each parts of a "modern" game engine, +like physics, audio, user inputs, files management, graphics (with DX in my case), etc. +The second one talk about the abstracts concepts of game engine like object memory management, +pointers good practice, game loop, and others importants things that we don't necessarily think about, +but which are necessary for the proper operation of a game engine. + +On top of that, you have the [online DX12 documentation][link_d3d12doc] which will go into greater +detail than F.D.Luna's book. Same advice for the Windows API to manage the window and the system messages. + +All these books have helped me a great deal in clarifying aspects of a game engine that I couldn't see +at first. But there's **no one right way to design a game engine**. Unlike a calculator program, where the +sole aim is to perform calculations as efficiently as possible, our engine must meet several needs in +terms of functionality to match the game we want to make. Most books and examples only give us one way +of doing things, although sometimes there's no other way than to go through this or that method to achieve +a task, the rest is up to our own judgement. + +> I really like the low-level programming because it's like a puzzle-game to make a game. + +My last piece of advice is to **take inspiration from existing engines**. And if the source code is available, +try to understand **why it's implemented the way it is**. Others previous developers have run into the same problems +you're about to encounter. For the time being, it would be a waste of time (and possibly of motivation) +to repeatedly get *our feet stuck in the carpet*. Above all, the aim of this blog is +to share successes and mistakes, that's what knowledge is all about. + +### It's (not) a trap! + +> This is a recurent chapter in a large part of my topics to describe, from my own experience +> and point of view, the points on which I may have found myself stuck, or which at some point +> required me to pause and reflect. +> +> Use them as you like! + +When I choose to use DX12 as my first API for a 3D engine, the price to pay for that is to take +even more time to complete the first draw on screen. That's part of the reason why it took me a +year to achieve this result. Today I'm on the 4th attempt/iteration, and that's not counting tests +on test code apart from the engine sources. I'll give you all details in upcoming posts! + +So why not using Vulkan if I wanted a challenge? Some reasons for that: **Vulkan isn't so well documented** +than DirectX, or at least it was easier to find DirectX docs and books. And it's for me a good +sign to avoid or to limit the *demotivation effect*: the more time and energy you invest in +something that doesn't bring you any satisfaction within the limits of your patience, the more +you give up or *make a break* on the project. The greater the challenge, the greater the risk +of *breaking*. This is normal, and it's through this kind of failure that we learn to know our +limits. So I keep Vulkan as a next feature or evolution for now. + + + +[link_gameeditor]: http://game-editor.com/static/Main_Page.html +[link_unity]: https://unity.com/fr +[link_godot]: https://godotengine.org/ +[link_boostlib]: https://www.boost.org/ +[link_cmake]: https://cmake.org/ +[link_conan]: https://conan.io/ +[link_git]: https://git-scm.com/ +[link_pix]: https://devblogs.microsoft.com/pix/download/ +[link_renderdoc]: https://renderdoc.org/ +[link_gameprogdx12]: https://amzn.eu/d/9aTUE79 +[link_holycpp]: https://amzn.eu/d/cN7pZ7Q +[link_gameenginearch]: https://amzn.eu/d/bEcBNXE +[link_gameprogpatterns]: https://gameprogrammingpatterns.com/ +[link_d3d12doc]: https://learn.microsoft.com/en-us/windows/win32/direct3d12/directx-12-programming-guide diff --git a/static/images/thumbnails/build_an_engine.jpg b/static/images/thumbnails/build_an_engine.jpg new file mode 100644 index 0000000..6c89c90 Binary files /dev/null and b/static/images/thumbnails/build_an_engine.jpg differ diff --git a/static/images/thumbnails/directx.jpg b/static/images/thumbnails/directx.jpg new file mode 100644 index 0000000..ce3ecea Binary files /dev/null and b/static/images/thumbnails/directx.jpg differ