A1 Scene Viewer

In this assignment you will build the core of a real-time scene viewer which renders scenes using the Vulkan API. This code serves as a foundation that you will modify, extend, and improve for the rest of the semester.

Scoring: this assignment is worth 15 points, total. Thanks to a convenient and intentional construction, each "point" is worth exactly 1% of your final course grade. Points are allocated as follows: A1-load is worth 2 points; A1-show is worth 3 points; A1-cull is worth 2 points; A1-move is worth 2 points; A1-hide is worth 3 points; A1x-fast is worth 1 point + up to 1 extra point; and A1-create is worth 2 points.

Points will be awarded for each section based on both the code and the sections of a report demonstrating and benchmarking the code.

Collaboration, Plagiarism, and Copyright

Reminder: this class takes a strict didactic, ethical, and legal view on copying code.

What to Do

Write code that can:

• A1-load Load a scene graph (a transformation hierarchy with associated meshes and cameras).
• A1-show Display the scene with a user-controlled camera, scene camera, or "debug" camera.
• A1-cull Perform frustum culling using bounding boxes.
• A1-move Animate the scene and camera.
• A1-hide Work in a "headless" mode which renders the scene without an active window (for benchmarking).
• Extra: A1x-fast develop and test potential performance improvements.

Use your creativity (and your code) to make something beautiful:

• A1-create Create an abstract 3D animation loop in scene'72 format for your code to render in real-time.

Demonstrate and test your code; write a report which includes screen recordings, images, timings, examples, graphs. This is non-trivial. See report template to understand what is required.

What to Turn In

Turn in your code in /afs/cs.cmu.edu/academic/class/15472-s24-users/<andrewid>/A1/. (If you are accessing AFS via an andrew machine, you may need to aklog cs.cmu.edu to acquire cross-realm tokens.)
Your turn-in directory should include:

• report/ report describing your code and illustrating that it works.
  • report/report.html start with the report template and replace the "placeholder" sections.
  • report/*.s72,*.b72 benchmarking scenes and data mentioned in your report.
  • report/* other files (images, animations) needed by your report.
• code/ the code you wrote for this assignment.
  • code/.git your code must be stored in a git repository with a commit history showing your development process.
  • code/Maekfile.js build script. When run as node Maekfile.js -- produces bin/viewer (bin/viewer.exe on Windows), your compiled scene viewer. You may wish to use maek and refer to how, e.g., Scotty3D or the 15-466 Example Code are set up. However, you may also write your own Maekfile.js.
  • Do not include any compilation products (e.g., executables, object files, or compiled SPIR-V).
  • Do not include any scene files (if it is a scene used as part of your report it should be in the report/ folder; the scene file for your animation should be in the animation/ folder.)
  • Do not include your report. Use a separate repository for your report if you want to version it.
• animation/ your created animation loop.
  • animation.s72 your main animation Scene'72 file.
  • *.b72 any data files needed by your scene.
  • animation.mp4 a screen recording (H.264 in MP4 container) of your animation playing in your viewer.

We expect your Maekfile.js to properly build your viewer on at least one of { Linux/g++; Windows/cl.exe; macOS/clang++ }. We will not penalize you for minor cross-platform compile problems; though we would appreciate it if you tested on Linux.

When we compile and run your code, we will set the Vulkan SDK environment variables as described in the LunarG SDK getting started guide. In addition, we will have GLFW installed system-wide via apt install libglfw3-dev.

We expect your report to be viewable in Firefox on Linux. You may wish to consult MDN to determine format compatibility for any embedded videos.

Command-Line Arguments

Some of the features you implement in this assignment are controlled by command-line arguments. Many of these are documented in more detail in the sections below. However, here is a list for reference:

• --scene scene.s72 (required) -- A1-load -- specifies the scene (in .s72 format) to view.
• --camera name (optional) -- A1-show -- view the scene through the camera named name. If such a camera doesn't exist in the scene, abort.
• --physical-device name (required/optional) -- A1-show -- use the physical device whose VkPhysicalDeviceProperties::deviceName matches name. If such a device does not exist, abort. You may choose how to deal with the flag not being specified, and how to allow the user to determine valid options (perhaps a --list-physical-devices flag?).
• --drawing-size w h (optional) -- A1-show -- set the initial size of the drawable part of the window in physical pixels. If the resulting swapchain extent does not match the requested size (e.g., because the window manager won't allow a window this requested size), abort. If this flag is not specified, do something reasonable, like creating a moderately-sized window.
• --culling none|frustum|... (optional) -- A1-cull, A1x-fast -- sets the culling mode. You may add additional culling modes when tackling extra goals.
• --headless events (optional) -- A1-hide -- if specified, run in headless mode (no windowing system connection), and read frame times and events from the file events. The flag --drawing-size is required when using headless mode, and specifies the size of the offscreen canvas that is rendered into.

You may add your own command line arguments; indeed, there is a section in the report template to document them.

Scene Format

This assignment (and subsequent assignments) will use scene'72 (.s72) format.

In this assignment, you may make the following simplifying assumptions about scene'72 files:

• There are no object reference cycles. (E.g., a node will never be its own descendant).
• Every camera object has exactly one instance in the scene. (This need not be the case for mesh or node objects, however!)
• No mesh object contains the "indices" property.
• All mesh objects use this "attributes" layout:

  "attributes":{
  	"POSITION":{ "src":"filename.b72", "offset":N+0,  "stride":28, "format":"R32G32B32_SFLOAT" },
  	"NORMAL":  { "src":"filename.b72", "offset":N+12, "stride":28, "format":"R32G32B32_SFLOAT" },
  	"COLOR":   { "src":"filename.b72", "offset":N+24, "stride":28, "format":"R8G8B8A8_UNORM" }
  }

A1-load Scene Data Structures and Loading

When launched with the command-line option --scene filename.s72, your viewer should load a scene in Scene'72 format from the file filename.s72. To complete this part of the assignment, you will both need to design a scene data structure to load the scene into and write parsing code to handle the scene format itself.

Suggestions:

• Don't worry about making a perfect or optimal JSON parser; instead, focus on getting simple files loading to the point you can actually extract the data. The example files don't exercise any of the corner case logic (string escapes) that make the JSON slightly tricky to load.
• You don't need to load into the same structs that you render. Consider loading into an intermediate data format and then translating this to whatever you end up doing for rendering.
• This goes extra for vertex buffers. A performant renderer will probably pool all of the mesh data for a scene in order to avoid pipeline and binding switches. (And this may also be easier to deal with than trying to dynamically create pipelines for different vertex layouts.)
• Focus on non-indexed meshes with a simple attribute stream format first. It will be easier to add support for different attribute data layouts and index formats once you have one working.
• Don't be afraid to generate your own test scenes.
• Consider ignoring animation data until later in the assignment process. It will be easier and more fun to debug animation loading once you can see your scene.

Warning: consider the Scene code from 15-466 as an anti-example! It only handles tree-structured scenes, isn't built for efficient traversal, and is pretty closely tied to OpenGL.

JSON handling inspiration: my favorite JSON handling library is, of course, the one I wrote; happy to chat about how that code works during office hours. My favorite JSON handling library I didn't write was a C library that implemented the JSON grammar with a compact state machine (with a transition table stored as a static array); I haven't discovered it again but if you do let me know.

A1-show Scene Display

Now that you've loaded the scene, it is time to show it.

This will involve traversing the scene and sending each mesh + transformation to the GPU using the Vulkan API. You should be able to start with your code from the Vulkan Tutorial. (Though you may consider using push constants rather than uniform buffers.)

This is a good time to add the --physical-device name and the --window-size w h command line flags to allow device selection and window sizing. (More detailed description above.)

Cameras

Your viewer should have three camera modes:

• in scene camera mode, the rendering happens through one of the cameras in the scene graph and the user cannot change the camera transformation (however, you should give them a way to cycle between scene cameras);
• in user camera mode, the rendering happens through a camera the user can move through the scene with keyboard+mouse control (I suggest either an orbit-style camera like Blender or a free-flying camera like, e.g., Descent);
• in debug camera mode, the rendering happens through a second user-controlled camera, but the culling happens for the previously-active camera (this is very useful for debugging culling). You may also want to add extra information to the debug camera view, like bounding boxes or the view frustum.

If started with the command-line option --camera name your viewer should launch in scene camera mode with the camera named name active. If the named camera does not exist in the scene, your viewer should print an error message and exit.

Otherwise, it is up to you to determine the details of how to activate and switch between scene, user, and debug camera modes. (Please document how in your report.) Some handy controls might include warping the user camera to the current scene camera and/or setting the debug camera to a position that can see the whole scene.

WARNING: Be aware that Vulkan uses a different convention for the normalized device coordinate axis directions and (in the case of z) ranges than OpenGL; be careful of this when writing code to build a projection matrix.

Materials / Lighting

We will not deal with materials or lighting in our scenes until A2. However, showing only vertex colors is quite "flat" looking, so display every scene as if lit by a hemispherical light in direction (0,0,1) with energy (1,1,1).

In other words, your fragment shader should do something along these lines:

	vec3 light = mix(vec3(0,0,0), vec3(1,1,1),
		dot(normal, vec3(0,0,1)) * 0.5 + 0.5);
	outColor = vec4(light * color, 1.0);

A1-cull Culling

The fastest triangle to render is the triangle you don't need to render. It's time to update your viewer to perform view frustum culling. When frustum culling is active, your code should check whether a mesh instance is visible before sending it to the GPU for drawing.

In order to perform a fast visibility test you should build a bounding volume for each mesh and check this volume against the viewing frustum. I suggest (but do not require) using bounding boxes as your bounding volume of choice; however, other bounding shapes might lead to improved performance in some situations.

When your viewer is run with the --culling mode command line option it will start with a given culling mode selected. The modes are as follows:

• --culling none -- disable culling. Every mesh gets drawn every frame, even if it is offscreen.
• --culling frustum -- check mesh bounding volumes vs the camera's viewing frustum. Any mesh whose bounding volume is outside the view frustum gets skipped.
• You may add additional options here if pursing A1x-fast improvements.

We are using a culling flag because, in your report, you will need to demonstrate that culling is working, and show scenes where it has both a positive and negative impact on frame rate.

Note: as mentioned above, when rendering with the debug camera, do culling as if rendering for the previously-selected (user or scene) camera. This can be very handy for checking to see if culling is actually working! (And demonstrating that it is working in your report.) You may also want to add some code to visualize the view frustum and/or bounding volumes for the meshes.

A1-move Animation

It is finally time to stop deferring animation. Update your parsing code to read "DRIVER" objects; add code to your main loop to track and update the current playback position for the animations; and patch in keyboard control to pause/unpause/restart the animations.

Note: the animation playback position for the first rendered frame should be 0, and should be advanced for subsequent frames by measuring the elapsed time from the first frame. (Or, in headless mode, using the times from the events file.)

Note2: your viewer should, by default, start with any scene animations playing and with the playback position at time 0. For the purposes of debugging/testing, you may wish to add command line options to start paused, set starting time, and/or loop animations.

A1-hide Headless Operation

For benchmarking, it will be very useful to have your application run exactly the same workload, and to run it as fast as possible in the background.

Add support for the --headless events command line flag. In headless mode, your code should not use any window system integration. Instead, it should create its own "swapchain" of the size defined by the --drawing-size command line option. Images in this swapchain will be made available for rendering by events in the events file (which also includes the frame times that should be reported to scene update code).

Important: events files contain timestamps to allow (e.g.) running animations at a repeatable rate; your code shouldn't try to delay to make these match wall-clock time.

Note: For reasons of stable timing, and to avoid mid-run failures, I suggest parsing the events file at startup and storing it into an events structure in memory.

Events File Format

The events file passed to --headless will be a UTF-8-encoded text file with '\n' (unix-style) line endings. Each line follows the same general format:

ts EVENT param1 ... paramN

Where ts is the time since the first frame as an integer number of microseconds (i.e., millionths of a second); EVENT is the event type (a string in all caps); and param1 through paramN are parameters (defined per event type).

Note: ts values will be nondecreasing; i.e., events are listed in chronological order.

Your code should support the following events:

• ts AVAILABLE -- make the least-recently-available image in the swap chain available to be rendered to (after waiting for any rendering pending on this image to complete). This, effectively, starts a frame rendering.
• ts PLAY t rate -- set animation playback time to t, a time in seconds, and the playback rate to rate. A rate of 0 is paused (time doesn't advance). A playback rate of 1 plays in real-time (relative to the timestamp values). You may choose to support (e.g.) fractional playback rates.
• ts SAVE filename.ppm -- save the most-recently-rendered image in the swap chain to filename.ppm in portable pixmap (binary/P6 variant). This event will never occur before a swap chain is made available.
• ts MARK description words -- your code should print the line "MARK description words" to stdout (useful for debugging and visualization).
• You may choose to specify additional event types. If you do, be sure to document them in your report.

Example Events File

Here is an example events file and the script that generated it. (Usage: node events.mjs > example.events.) You may find yourself making similar files when doing the performance testing part of your report.

A1x-fast Extra: Go Fast

Note: this section is work 1 point + up to 1 extra point. I tagged it as "extra" because you can skip it entirely and still receive 14/15 = ~93% on the assignment. On the other hand, if you try (and document) several substantial approaches to improving your viewer's rendering speed you can earn some extra credit here.

Within the framework of this basic viewer there are a ton of ways to (attempt to) make your code go faster. The final segment of this assignment is an open-ended exploration of these possibilities.

To receive one point on this segment, you may add any of the following to your system (and document the impact in your report):

• Use a BVH, spatial grid, or octree to accelerate frustum culling. Be aware of losing time to structure re-builds to animation!
• Transform loaded meshes into indexed meshes and/or triangle strips to reduce GPU memory usage and accelerate vertex processing.
• Flatten parts of the scene into static geometry buffers to reduce CPU overhead.
• Optimize your mathematical operations and/or scene memory layout to reduce GPU usage.
• Parallelize scene graph traversal / culling on the CPU.
• Move more of scene graph evaluation, animation, and/or frustum culling to the GPU.

To receive up to one extra point, go beyond by attempting more items from the list above or developing and documenting other substantial improvements of your own devising.

Don't forget the report

Don't forget to write the report.