Porting Samples to Mac

On October 22, 2013, Apple released OS X Mavericks, also known as OS X version 10.9. This version of the operating system included a long awaited update to the supported version of OpenGL. The 6th edition of the OpenGL SuperBible is about OpenGL version 4.3, and unfortunately, Apple’s latest and greatest only supports version 4.1 of the API. As OpenGL 4.1 was released on July 26, 2010, this puts OS X more than three years behind. However, not all of the book’s samples make use of all of the latest features, and it’s possible to run many of them on version 4.1 of the API. I’ve ported what I can.

What Works, What Doesn’t

Now, there are clearly some samples that aren’t going to run on OpenGL 4.1. For instance, anything that uses compute shaders (the depth-of-field and flocking samples, for example), shader storage buffers (the fragmentlist sample), atomic counters or image loads and stores isn’t going to port. Luckily, although the Mac OpenGL implementation is only version 4.1, it does have a couple of extensions from OpenGL 4.2 — GL_ARB_internalformat_query and GL_ARB_texture_storage, the latter of which is used by the sb6 framework’s texture loader.

To be fair, the book’s preface does say that the samples have been tested on Mac. We’ve received a couple of poor reviews on Amazon because the code isn’t usable on Mac. For my day job, I’m the architect of AMD’s PC OpenGL driver, which we use for Windows and Linux. My cube neighbor is the architect for AMD’s Mac driver. Apple can be pretty secretive about what’s coming up and when, and all I had to go on when I wrote that preface (back in January of 2013) was insider knowledge of what my neighbor was working on. Even he doesn’t know what’s going to be in the next OS version or when it might release. However, it seemed that he was making pretty good progress and that I’d have a shot at building Mac versions of the samples by the time the book shipped. So much for my ability to predict the future.

The Environment

Unlike Windows and Linux, where the hardware vendor ships drivers which determine the version of OpenGL available on your machine, on the Mac platform, big chunks of the OpenGL stack are part of the operating system and so an OS update is required to bump the OpenGL version. Hence, although AMD, NVIDIA and Intel have all had OpenGL 4.x capable hardware shipping in Mac platforms for some time, we needed to wait until the 10.9 Mavericks update to actually expose it. Step 1, then, is to update the operating system to the latest available. After a quick stop off at the App Store, and a rather large, 5GiB free download (which is excellent), I was installing OS X 10.9 Mavericks on my trusty Mac Book Pro.

OS X 10.9 on Mac Book Pro

OS X 10.9 on Mac Book Pro

Of course, the next step is to install XCode, which includes all the developer tools. What’s more to say about XCode? It seems that people either love it or hate it. I wouldn’t say I love it. I’ve used a lot of development environments in the past. While I’m most comfortable with Visual Studio, I’ve had spells with Borland’s C++ Builder IDE, KDevelop, Anjuta, Eclipse, and Qt Creator. XCode is different and I would imagine would take a lot of getting used to. I was able to coerce CMake into generating XCode project files for the samples. But, once loaded into XCode, getting things to build was close to impossible. It was difficult to navigate the project, difficult to see what was going wrong, and an extremely frustrating experience.

First Pass at SuperBible 6 Samples in Xcode

First Pass at SuperBible 6 Samples in Xcode

In the end, I opted to generate regular Makefiles at the command prompt. This seemed much more straight forward, and after a short time, I had everything building and linking. Upon running the first example, it crashed almost immediately. Now, this may be an artifact of the version of GLFW that I’ve used for the projects, but it seems that if you ask for GLFW_OPENGL_VERSION_MAJOR as 4 and GLFW_OPENGL_VERSION_MINOR as 1 (i.e., ask for OpenGL 4.1), context creation will fail. However, if you set GLFW_OPENGL_VERSION_MAJOR to 3 and GLFW_OPENGL_VERSION_MINOR to 2 (ask for OpenGL 3.2), you’ll get a 4.1 context. You also need to ask for a forward compatible context.

Debugging this was a pain. XCode wouldn’t cooperate — I couldn’t get it to debug anything it didn’t build, and I couldn’t get it to build my samples. Time to whip out the trusty command line debugger, gdb. Except, with Mavericks, gdb is gone and now we have lldb. lldb seems good. I’m not quite sure why a whole new debugger is necessary. I’m certain that lldb does some pretty neat things that gdb can’t do, but for the things that are the same (setting breakpoints, for example), it seems to be different (and therefore unfamiliar) for the sake of it. Learning a new debugger just to step through sample applications I know already work was yet more pain.

Porting

For the most part, modifying the #version declaration in the shaders to request version 410 core rather than 430 was sufficient to get the samples working. Of course, dropping the shading language version meant that I couldn’t make use of some of the convenience functions that have been incorporated recently, such as some forms of implicit conversion and promotion from scalar to vector. I fixed the most trivial of these and was fairly quickly up and running with the first few samples.

OpenGL SuperBible Samples Running on Mac OS X 10.9

OpenGL SuperBible Samples Running on Mac OS X 10.9

The biggest issue was the lack of explicit binding declarations on sampler uniforms in GLSL. These allow you to write, directly in the shader code, the texture unit that a sampler corresponds to. The type of declaration I’m referring to looks like this:

layout (binding = 0) uniform sampler1D grasspalette_texture;
layout (binding = 1) uniform sampler2D length_texture;
layout (binding = 2) uniform sampler2D orientation_texture;
layout (binding = 3) uniform sampler2D grasscolor_texture;
layout (binding = 4) uniform sampler2D bend_texture;

Here, I’ve assigned the five sampler uniforms used by the shader to the first five texture units. I could have left gaps or assigned the uniforms in any order. By default, sampler uniforms are associated with texture unit zero. For simpler examples with that use only a single texture, I can remove the binding layout qualifiers and take that zero default. For more complex examples such as the one above, I’d have to query the locations of the sampler uniforms and set their values with glUniform1i from the application. The binding qualifiers allow the application code to be more concise and the shader code to be more explicit. Therefore, I won’t be modifying any of the examples that make extensive use of this OpenGL 4.2 feature.

The second feature that most of the samples rely on is part of the GL_ARB_base_instance extension, which is part of OpenGL 4.2. This extension adds variants of several instanced draw commands which include a baseinstance parameter. These functions are fairly trivial additions to the API, with the additional parameter not requiring any validation and being passed directly to the GPU as part of the draw command. It’s surprising that Apple wasn’t able to squeeze this one into the Mavericks update. Unfortunately, the sb6 object loader uses the glDrawArraysInstancedBaseInstance and glDrawElementsInstancedBaseVertexBaseInstance functions to render objects.

To work around this issue, I’ve modified the object rendering code such that on Mac, it calls the variants of the functions without the baseinstance parameter. They’ll behave as if the baseinstance parameter was set to zero, which will be correct in the vast majority of the examples. In addition to the new APIs, this extension defines the meaning of the last parameter of the structures used by glDrawArraysIndirect and glDrawElementsIndirect functions, which is undefined in GL_ARB_draw_indirect. However, the only sample that makes use of this parameter is the asteroids sample which demonstrates GL_ARB_multi_draw_indirect, which is also unsupported in Mavericks.

Errors and Debugging

The OpenGL SuperBible sample code framework (referred to as sb6) is pretty simple. It’s not really designed as an experimentation environment or as a full-featured utility library. Rather, it’s a good place to stuff boilerplate code in order to make the actual sample application more concise. As such, it, and the sample applications do little-to-no error checking. This is fine — the examples that shipped all work and don’t need to be debugged. In the dark ages, people would hack together framework code to call glGetError 1 after every OpenGL function call. We’ll have none of that in our applications. Instead, we rely on the ARB_debug_output extension.

GL_ARB_debug_output was promoted to core OpenGL in version 4.3, but has been available as an ARB extension since 2010, and as a vendor specific AMD extension prior to that. This extension has even been promoted to KHR status to allow it to be exposed by OpenGL ES implementations consistently with desktop implementations. Strangely, while OS X 10.9 appears to expose GL_EXT_debug_label and GL_EXT_debug_marker extensions (which are presumably used by some of the developer tools), it does not expose GL_ARB_debug_output nor GL_KHR_debug, which basically means that the framework’s debugging code is disabled.

With the presence of GL_ARB_debug_output (or version 4.3 of OpenGL, where it is a core feature), the OpenGL implementation will call a user-supplied callback function whenever an error is generated. By turning on synchronous output, one can insert a breakpoint inside this callback function and see the exact line of code and function call that generated the error. The error message passed to the callback function contains detailed information about what caused the error. This is the way to debug OpenGL applications.

Other Notable Omissions

There’s a few more conspicuously absent features in the Mavericks OpenGL implementation. As the ARB has produced new features in OpenGL, we’ve gone to some effort to document them not only in the core specification, but as extensions as well. This allows vendors and implementers to ship a baseline OpenGL version, but cherry-pick a couple of handy features on top of them. As I mentioned, Apple has done this with GL_ARB_internalformat_query and GL_ARB_texture_storage, but there’s nothing more beyond the core features of OpenGL 4.1 and a couple of vendor extensions.

Mac OS X 10.9 Mavericks in OpenGL Extensions Viewer

Mac OS X 10.9 Mavericks in OpenGL Extensions Viewer

One example is the GL_ARB_multi_draw_indirect extension. A core feature of OpenGL 4.3, it’s actually pretty simple. GL_ARB_draw_indirect introduced a way to send a draw to OpenGL with its parameters stored in a buffer object. The multi version of this simply adds a loop around it allowing a whole bunch of draws to be sent in one go (with glMultiDrawArraysIndirect and glMultiDrawElementsIndirect). In their most na├»ve form, they can be implemented in the very top layers of an OpenGL driver using a simple loop around the underlying non-multi versions of the functions. However, it’s actually pretty trivial to send the iteration count and stride parameters to the GPU and let it handle the loop.

Other interesting omissions include GL_ARB_map_buffer_alignment, which simply guarantees that pointers returned from glMapBuffer and glMapBufferRange will be aligned to at least a 64-byte boundary (which is important if you want to process data with some SIMD implementations), GL_ARB_invalidate_subdata, which can be implemented as a hint, and GL_ARB_texture_storage_multisample, which simply extends GL_ARB_texture_storage (which is supported) to multisample texture targets.

Summary

Getting OpenGL 4.3 samples running on an OpenGL implementation that is three years out of date at launch is a pain. It’s not insurmountable, but it certainly can be challenging. If developing an engine from scratch that had to support it, it’s certainly possible to target OpenGL 4.1 as a baseline and either compile-time or run-time detect newer versions of OpenGL to enable more up-to-date features on platforms that support them. However, the 6th edition of the OpenGL SuperBible is a book about OpenGL 4.3, not OpenGL 4.1. Simply not using features introduced to OpenGL in the last three years defeats the purpose of the updated edition. Regardless, I’ve managed to get many of the OpenGL SuperBible samples running on Mac OS X 10.9.

The major part of the fixes for Mac were pushed to the master repository on Github over the weekend. The downloadable package has also been updated and is available from the example code page on this site.

Notes:

  1. … and sometimes even leave it on in the shipping application!
Mac, OpenGL , ,

10 responses to Porting Samples to Mac


  1. Christopher

    Thank you for the explanation on why you didn’t port back the remaining examples and also provided (in this blog) tips for how to make them work on 4.1 :)

  2. Brian Jones

    The effort is much appreciated. I’m on Mavericks, and I got some of the example to work – Julia fractals works, but Tessellatedcube just shows a green screen, and many other crash. Not to trivialize you work on the mac side, but there is always recommending users to bootcamp. I have windows 8.1 on an early 2013 mbp with a gt650m, and it has full OpenGL 4.4 support. Apple is just lazy. They couldn’t even give us arg explicit uniform location. *sigh* another 2 years before we get compute shaders. Oh wait! *Restarts computer and holds down alt/option key* There we go, but still *sigh*

  3. Moa

    Great post. Very informative. I’m working on a modern cross-platform jet flight simulator in JoGL+OpenGL (2.1, at the moment, but moving to 4.1 now that Mavericks has moved).

    I have noticed that the Mac will crash in the driver (as in, the machine resets, not just an application crash) after a few minutes when running code that runs for hours on Windows and Linux. I wish Apple would let folks like you write their OpenGL drivers.

    Fingers crossed this excellent book sells well and you can justify upgrading your MacBook to the missus. I’ve just got one of the new MacBooks with the NVidia 750M and 2GB of VRAM, for the principal purpose of developing with OpenGL. I’m sure one would site well on your desk too :)

    C’mon Apple, get with the program! upgrade your OpenGL, please.

  4. Moa

    Oh, while I remember. For years I have used and advocated ATI/AMD hardware. I find the drivers to produce better output than NVidia (not to start a flamewar, but this is my observation at least for the programs I use and create).

    However, I’ve just moved to an NVidia GTX 780 on my desktop. I find the programming the NVidia to be more tricky, in that it will accept dodgy input and crash, rather than the AMD drivers which seem to be better at detecting when I screw up and produce sensible output.

    Why did I make this switch? two reasons:
    1) On Windows, I was never sure whether and upgrade to the AMD DirectX drivers would cause issues or not. Every four months there’d be a release that seemed to work for everything I used, but intermediate releases could be problematic.

    2) On Linux the NVidia driver installed is *vastly* superior to the AMD one. I don’t mean the drivers themselves, I mean the installer (and control panel, incidentally, mult-monitor is hard with AMD under Linux). Upgrading the AMD drivers to new versions under Linux is even less pleasant.

    I mean this as constructive criticism, and not as an ungrateful gripe. Some love to the AMD driver installer would be nice – and tracking Ubuntu as a first class target soon after releases (although I use Mint, but it’s all the same) would do wonders to ease of use. We don’t even need a graphical installer, just an APT package would be fabulous (although keeping the graphical Catalyst Control Panel would be nice).

    I like AMD hardware. I like programming to the AMD OpenGL drivers. Making device driver updates easier (especially on Linux) would get me buying and advocating AMD again. Just thought you might like one little data point (hopefully you can wave this in front of your boss to justify spending effort on making AMD a first class experience on Linux, and better than NVidia :) ).

    Thanks for all your hard work on Mac and Linux, and getting the SuperBible examples to work there. This blog which is very informative.

  5. Polar

    The reason that asking for an OpenGL 3.2 context gives a 4.1 context if your Mac supports it is because Apple now only offers a compatibility (2.1) or a core (3.3 or 4.1 depending on your Mac), but the core profile still uses the numbered constant.

  6. M. Jeragh

    Thank you for your Book, Mac support and this blog. While going through the Mac samples, I get the following errors(I can comment out stopwatch file, but I don’t know what to do with the identifiers.)

    /Downloads/MacChapterSamples/GLUTCore32Demo/GLUTCore32Demo/Glut32CoreDemo.mm:12:10: ‘StopWatch.h’ file not found

    /Downloads/MacChapterSamples/StonehengeModel/GLString.mm:219:16: Use of undeclared identifier ‘GL_TEXTURE_BIT’
    /Downloads/MacChapterSamples/StonehengeModel/GLString.mm:221:18: Use of undeclared identifier ‘GL_TEXTURE_RECTANGLE_EXT’
    /Downloads/MacChapterSamples/StonehengeModel/GLString.mm:223:20: Use of undeclared identifier ‘GL_TEXTURE_RECTANGLE_EXT’
    /Downloads/MacChapterSamples/StonehengeModel/GLString.mm:225:20: Use of undeclared identifier ‘GL_TEXTURE_RECTANGLE_EXT’
    /Downloads/MacChapterSamples/StonehengeModel/GLString.mm:226:20: Use of undeclared identifier ‘GL_TEXTURE_RECTANGLE_EXT’
    /Downloads/MacChapterSamples/StonehengeModel/GLString.mm:227:17: Use of undeclared identifier ‘GL_TEXTURE_RECTANGLE_EXT’
    /Downloads/MacChapterSamples/StonehengeModel/GLString.mm:229:3: Use of undeclared identifier ‘glPopAttrib’

    Please Help.

  7. Tyler

    Hey, I just bought SuperBible 6th and downloaded the newest example code.

    I use Mac OS Mavericks 10.9.1
    However very few examples work.

    Could you please test on 10.9.1? Really appreciate that.

    Best regards,
    Tyler

  8. Tyler

    Sorry it is not a problem with 10.9.1

    It seems that some GLSL’s version is left unchanged.

    I just changed that of singletri from 420 to 410 and it works

    Best regards,
    Tyler

  9. Smith Richardson

    The Key Events in your StonehengeCGL fullscreen OpenGL OS X application fail to work (from OpenGL SuperBible 6th Ed). I have only tested it on OS X version 10.9. I have read all the Apple documentation and still I am unable to locate the issue so thought you may have a quick solution or answer. Also, I really enjoyed reading your book.

    Thank you,
    Smith Richardson

  10. Bernardo

    Whenever I try to remake the source code, on 10.9.1 the I get a lot of error messages.

    most of my error messages are :
    Undefined symbols for architecture x86_64:
    “sb6IsExtensionSupported(char const*)”, referenced from:
    sb6::application::run(sb6::application*) in polygonsmooth.o

    Am I missing something? I linked the opengl framework and glfw framework. Added the header search path where the sb6.h and include folder of glfw.

Leave a Reply

Your email address will not be published. Required fields are marked *


9 + = 14

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>