A1 Scene Viewer

In this assignment you will build the core of a real-time scene viewer which renders scenes using the Vulkan API. This code serves as a foundation that you will modify, extend, and improve for the rest of the semester.

Scoring: this assignment is worth 15 points, total. Thanks to a convenient and intentional construction, each "point" is worth exactly 1% of your final course grade. Points are allocated as follows: A1-load is worth 2 points; A1-show is worth 3 points; A1-cull is worth 2 points; A1-move is worth 2 points; A1-test is worth 3 points; A1x-fast is worth 1 point + up to 1 extra point; and A1-make is worth 2 points.

Points will be awarded for each section based on both the code and the sections of a report demonstrating and benchmarking the code.

Collaboration, Plagiarism, and Copyright

Reminder: this class takes a strict didactic, ethical, and legal view on copying code.

What to Do

We grade on evidence, so -- as you complete the tasks described below -- you also need to create a report that proves you have completed them successfully. This report should include screen recordings, images, timings, example scenes, and graphs sufficient to convince us that you have completed the tasks.

The report should be written in HTML in the style of the report template. Do not use a fancy HTML editor, just use a text editor.

Starting with your codebase from the nakluV tutorial, write code that can:

• A1-load Load a scene graph (a transformation hierarchy with associated meshes and cameras).
• A1-show Display the scene with a user-controlled camera, scene camera, and "debug" camera.
• A1-cull Perform frustum culling using bounding boxes.
• A1-move Animate the scene and camera.
• Extra: A1x-fast develop potential performance improvements.

Stress-test your code:

• A1-test build scenes that move the performance bottleneck between the CPU and GPU.

Use your creativity (and your code) to make something beautiful:

• A1-make Create an abstract 3D animation loop in scene'72 format for your code to render in real-time.

A final reminder: developing the report is a non-trivial part of the assignment. Do not neglect it. Build it as you build your code.

What to Turn In

Turn in your code in /afs/cs.cmu.edu/academic/class/15472-s26/<andrewid>/A1/. (If you are accessing AFS via an andrew machine, you may need to aklog cs.cmu.edu to acquire cross-realm tokens.)
Your turn-in directory should include:

• report/ report describing your code and illustrating that it works.
  • report/report.html start with the report template and replace the "placeholder" sections.
  • report/*.s72,*.b72 benchmarking scenes and data mentioned in your report.
  • report/* other files (images, animations) needed by your report.
• code/ the code you wrote for this assignment.
  • code/.git your code must be stored in a git repository with a commit history showing that you used version control throughout your development process.
  • code/Maekfile.js build script. When run as node Maekfile.js produces bin/viewer (bin/viewer.exe on Windows), your compiled scene viewer.
  • Do not include any compilation products (e.g., executables, object files, or compiled SPIR-V).
  • Do not include any scene files (if it is a scene used as part of your report it should be in the report/ folder; the scene file for your animation should be in the animation/ folder.)
  • Do not include your report. Use a separate repository for your report if you want to version it.
• animation/ your created animation loop.
  • animation.s72 your main animation Scene'72 file.
  • *.b72 any data files needed by your scene.
  • animation.mp4 a screen recording (H.264 in MP4 container) of your animation playing in your viewer.

We expect your Maekfile.js to properly build your viewer on at least one of { Linux/g++; Windows/cl.exe; macOS/clang++ }. We will not penalize you for minor cross-platform compile problems; though we would appreciate it if you tested on Linux.

We will compile and run your code in an environment set up to build the nakluV tutorial -- GLFW 3.4 and the Vulkan SDK will be available.

We expect your report to be viewable in Firefox on Linux. You may wish to consult MDN to determine format compatibility for any embedded videos.

Check-In

For the Check-In, present evidence (in the appropriate thread in course Zulip) that you have completed A1-load and A1-show. You may re-use this evidence in your final report.

Completing the check-in is worth 2pts, one point each toward A1-load and A1-show.

Allowed Libraries

Although one of this class's principles is "No Magic," we cannot open every black box; so you are allowed to use a few approved libraries for specific tasks in your code. (These libraries are in addition to GLFW and the Vulkan SDK which -- as we note above -- will be available in our build environment.)

• Vector Math: GLM -- though note that GLM is OpenGL-focused, so be careful with handedness and depth range of perspective matrices. Also, at least one student has reported that writing their own code was faster than using GLM's implementations for some vector operations.
• JSON Parsing: sejp -- and our provided example loader code.
• Texture Loading: stb_image

If you use any of these libraries, you will need to include the relevant code in your turn-in repository. We suggest a git submodule.

If you would like to use a different library for one of these tasks you do need approval from course staff (there will be a Zulip thread). Approval may not be given for certain libraries, and may take some time.

Command-Line Arguments

Some of the features you implement in this assignment are controlled by command-line arguments. Many of these are documented in more detail in the sections below. The "optional" and "required" tags indicate whether the argument is required on the command line, not whether it is required that you implement it.

• --scene scene.s72 (required) -- A1-load -- specifies the scene (in .s72 format) to view.
• --camera name (optional) -- A1-show -- view the scene through the camera named name. If such a camera doesn't exist in the scene, abort.
• --physical-device name (required/optional) -- A1-show -- use the physical device whose VkPhysicalDeviceProperties::deviceName matches name. If such a device does not exist, abort. You may choose how to deal with the flag not being specified, and how to allow the user to determine valid options (perhaps a --list-physical-devices flag?).
• --drawing-size w h (optional) -- A1-show -- set the initial size of the drawable part of the window in physical pixels. If the resulting swapchain extent does not match the requested size (e.g., because the window manager won't allow a window this requested size), abort. If this flag is not specified, do something reasonable, like creating a moderately-sized window.
• --culling none|frustum|... (optional) -- A1-cull, A1x-fast -- sets the culling mode. You may add additional culling modes when tackling extra goals.
• --headless (optional) -- A1-test -- if specified, run in headless mode (no windowing system connection), and read frame times and events from standard input. In headless mode, the flag --drawing-size specifies the size of the offscreen canvas that is rendered into.

You may also add your own, additional, command line arguments; indeed, there is a section in the report template to document them.

Scene Format

This assignment (and subsequent assignments) will use scene'72 (.s72) format.

In this assignment, you may make the following simplifying assumptions about scene'72 files — though your code should strive to fail gracefully if they are violated (e.g., by ignoring/skipping extra data instead of crashing):

• There are no object reference cycles. (E.g., a Node will never be its own descendant).
• Every Camera object has exactly one instance in the scene.
• No Mesh object contains the "indices" property.
• All Mesh objects have a "topology" of "TRIANGLE_LIST".
• All Mesh objects use exactly these "attributes", with these "format"s and "stride"s (different "src" file[s] and "offset"s may appear):

  "attributes":{
  	"POSITION": { "src":"cube.b72", "offset":0,  "stride":48, "format":"R32G32B32_SFLOAT" },
  	"NORMAL":   { "src":"cube.b72", "offset":12, "stride":48, "format":"R32G32B32_SFLOAT" },
  	"TANGENT":  { "src":"cube.b72", "offset":24, "stride":48, "format":"R32G32B32A32_SFLOAT" },
  	"TEXCOORD": { "src":"cube.b72", "offset":40, "stride":48, "format":"R32G32_SFLOAT" },
  }

• All Material objects have brdf "lambertian" and any normal or displacement maps can be ignored.
• All texture files will be loadable by stb_image (specifically: png or jpeg format).
• The scene will contain at most two Lights: one or zero "sun"-type Lights with angle 0 (i.e., a pure directional light); and one or zero "sun"-type Lights with angle approximately 3.1415926 (i.e., a full hemisphere light). This means that the same lighting setup used in the nakluV tutorial will be sufficient for your scene.
• Your code may ignore any Environment objects in the scene.
• Textures may be in "srgb" or "linear" format, but not "rgbe" format.
• All objects have non-empty names.

A1-load Scene Data Structures and Loading

When launched with the command-line option --scene filename.s72, your viewer should load a scene in Scene'72 format from the file filename.s72.

Suggestions:

• We provide example code which uses sejp to parse an s72-format scene into an intermediate C++ data structure. You may use, modify, or ignore this code. If you ignore our example code, consider still using sejp; don't write your own JSON parser unless you really, really want to.
• You don't need to load into the same structs that you render. Consider loading into an intermediate data format (or doing so with our example code) and then translating this to whatever you end up doing for rendering.
• This goes extra for vertex buffers. A performant renderer will probably pool all of the mesh data for a scene in order to avoid pipeline and binding switches. (And this is also easier to deal with than trying to dynamically create pipelines for different vertex layouts.)
• Note that the assumptions above allow you to focus on non-indexed meshes and a single data stream format. Don't try to write a solution which works for all attribute layouts and/or indexed meshes until you need it.
• Don't be afraid to generate your own test scenes.
• Consider ignoring animation data until later in the assignment process. It will be easier and more fun to debug animation loading once you can see your scene.

Warning: consider the Scene code from 15-466 as an anti-example! It only handles tree-structured scenes, isn't built for efficient traversal, and is pretty closely tied to OpenGL.

A1-show Scene Display

Now that you've loaded the scene, it is time to show it.

This will involve traversing the scene and sending each mesh + transformation to the GPU using the Vulkan API. You should be able to start with your code from the nakluV Tutorial, which is already set up for drawing object instances.

Cameras

Your viewer should have three camera modes:

• in scene camera mode, the rendering happens through one of the cameras in the scene graph and the user cannot change the camera transformation (however, you should give them a way to cycle between scene cameras);
• in user camera mode, the rendering happens through a camera the user can move through the scene with keyboard+mouse control, as per nakluV Step 3a;
• in debug camera mode, the rendering happens through a second user-controlled camera, but the culling happens for the previously-active camera (this is very useful for debugging culling). When using the debug camera, your renderer should display object bounding boxes and camera frustums using lines.

If started with the command-line option --camera name your viewer should launch in scene camera mode with the camera named name active. If the named camera does not exist in the scene, your viewer should print an error message and exit.

Otherwise, it is up to you to determine the details of how to activate and switch between scene, user, and debug camera modes. (Please document how in your report.) Some handy controls might include warping the user camera to the current scene camera; setting the debug camera to a position that can see the whole scene; and toggling the debug bounding box / frustum lines.

Materials / Lighting

As per the assumptions above, all materials in your scene will be lambertian, and the only lights in the scene will be a distant directional light and/or a hemisphere light.

Therefore, your fragment shader will probably do something like this:

	vec3 energy =
		  SKY_ENERGY * (dot(n, SKY_DIRECTION) * 0.5 + 0.5)
		+ SUN_ENERGY * max(dot(n, SUN_DIRECTION), 0.0);
	outColor = vec4(texture(ALBEDO, texCoord) / 3.1415926 * energy, 1.0);

A1-cull Culling

The fastest triangle to render is the triangle you don't need to render. It's time to update your viewer to perform view frustum culling. When frustum culling is active, your code should check whether a mesh instance is visible before sending it to the GPU for drawing.

In order to perform a fast visibility test you should build a bounding volume for each mesh and check whether this volume intersects the viewing frustum. Use local-axis-aligned bounding boxes for your first implementation; you may test other shapes later for potential performance improvement.

When your viewer is run with the --culling mode command line option it will start with a given culling mode selected. The modes are as follows:

• --culling none -- disable culling. Every mesh gets drawn every frame, even if it is offscreen.
• --culling frustum -- check mesh bounding boxes vs the camera's viewing frustum. Any mesh whose bounding box is outside the view frustum gets skipped.
• You may add additional options here if pursing A1x-fast improvements.

We are using a culling flag because, in your report, you will need to demonstrate that culling is working, and show scenes where it has both a positive and negative impact on frame rate.

Note: as mentioned above, when rendering with the debug camera, do culling as if rendering for the previously-selected (user or scene) camera. This can be very handy for checking to see if culling is actually working! (And demonstrating that it is working in your report.)

Note2: implement a complete box-frustum intersection test (whether via separating axis or some other method of your choosing). Think about what the hard cases for such an intersection test are and how to show -- in your report -- that you've handled these cases. Remember that it's okay to discuss such things with your classmates; credit them if you do.

A1-move Animation

It is finally time to stop deferring animation. Update your parsing code to read "DRIVER" objects; add code to your main loop to track and update the current playback position for the animations; and patch in keyboard control to pause/unpause/restart the animations.

Note: the animation playback position for the first rendered frame should be 0, and should be advanced for subsequent frames by measuring the elapsed time from the first frame. (Or, in headless mode, using the times from the AVAILABLE events.)

Note2: your viewer should, by default, start with any scene animations playing and with the playback position at time 0. For the purposes of debugging/testing, you may wish to add command line options to start paused, set starting time, and/or loop animations.

A1-test Bottleneck Hunting

This part of the assignment depends on headless operation, as per nakluv Step 6a. That is, when the --headless command line flag is passed, your code should read AVAILABLE <dt> [filename.ppm] events from standard input, advance time by dt, render a frame, and save that frame to filename.ppm (if a filename is given).

Your goal in this section is to develop scenes that bottleneck in your renderer on either the CPU or the GPU.

Your CPU-bottlenecked scene should show clear performance sensitivity to a CPU-based part of the rendering workload. You will need to demonstrate this in your report by showing that performance changes when you adjust some parameter of your scene that increases CPU work without influencing GPU work.

Your GPU-bottlenecked scene should have performance sensitive to a GPU-based part of the rendering workload. You will need to demonstrate this in your report by showing that performance changes when you adjust some part of your scene that doesn't change CPU work but does change GPU work.

Any attempt on this section -- make some scene and collect some data -- will earn you 1pt. For a full 3pts of credit, you must successfully demonstrate both a CPU-bottlenecked scene [+1pt] and a GPU-bottlenecked scene [+1pt].

Note: we are intentionally vague about what sort of test scenes may work here. We will brainstorm in class, and you are also encouraged to discuss with your classmates. Please give credit in your report.

Note: there are at least three ways to generate a scene parametrically -- you could build the scene directly in your viewer using a custom command-line option; you could write a script to build the scene in Blender and then use the s72 exporter; or you could write a script to create a custom s72 file. If you choose to write code that makes a scene directly in your viewer's internal representation, please make sure that the scene could have been represented in standard s72 format -- do not, for example, have a special very expensive material type that runs a time-wasting shader.

A1x-fast Extra: Go Fast

Note: this section is work 1 point + up to 1 extra point. I tagged it as "extra" because you can skip it entirely and still receive 14/15 = ~93% on the assignment. On the other hand, if you try (and document) several substantial approaches to improving your viewer's rendering speed you can earn some extra credit here.

Within the framework of this basic viewer there are a ton of ways to (attempt to) make your code go faster. The final segment of this assignment is an open-ended exploration of these possibilities.

To receive one point on this segment, you may add any of the following to your system (and document the impact in your report):

• Use a BVH, spatial grid, or octree to accelerate frustum culling. Be aware of losing time to structure re-builds to animation!
• Transform loaded meshes into indexed meshes and/or triangle strips to reduce GPU memory usage and accelerate vertex processing.
• Flatten parts of the scene into static geometry buffers to reduce CPU overhead.
• Optimize your mathematical operations and/or scene memory layout to reduce GPU usage.
• Parallelize scene graph traversal / culling on the CPU.
• Move more of scene graph evaluation, animation, and/or frustum culling to the GPU.

To receive up to one extra point, go beyond by attempting more items from the list above or developing and documenting other substantial improvements of your own devising.

Don't forget the report

Don't forget to write the report.