Open-sourcing Imageworks BSDF library (#1986)

Sony Pictures Imageworks BSDF library (working title BSDL) is here
published and introduced to testrender. It is a header only library
where everything is inlined and can be used on GPU. Although we haven't
kept the BSDFs up to date with the latest research in the last couple of
years, I hope you, at least, like the infrastructure. There are three
main items in this PR
  1. Put BSDL as an INTERFACE cmake library in src/libbsdl.
  2. Introduce the switch/case automatic pseudo virtual methods in
     testrender to handle all the existing BSDFs.
  3. As an example integrate SPI's ThinLayer BSDF in testrender and add
     a test for it.

BSDL is a self-contained lib depending only on Imath. There is not even
a need to link to OSL and it can be used as a build only dependency.

Inside libbsdl directory there is a README with further information, but
documentation is the usual weak point with production code. We use this
code as-is directly for making movies with our in-house render, and we
welcome everybody to contribute theirs. The name "BSDL" is one of my
half-broken name ideas and I'm very much open to suggestions.

I recommend to first look at my changes to testrender, integration was
very easy. Then dive in the internals if you want. Credits for this code
go mainly to Chris Kulla and me (Alejandro Conty), but there are also
other smaller contributions from other Imageworks members, including
alumni Laura Lediaev who wrote the LTC sheen implementation.

Signed-off-by: Alejandro Conty <aconty@imageworks.com>

Author: aconty

CMakeLists.txt

@@ -250,6 +250,7 @@ add_subdirectory (src/liboslcomp)
 add_subdirectory (src/liboslquery)
 add_subdirectory (src/liboslexec)
 add_subdirectory (src/liboslnoise)
+add_subdirectory (src/libbsdl)
 add_subdirectory (src/oslc)
 add_subdirectory (src/oslinfo)
 

src/cmake/cuda_macros.cmake

@@ -15,7 +15,7 @@ if (CUDA_NO_FTZ)
 endif ()
 
 # Compile a CUDA file to PTX using NVCC
-function ( NVCC_COMPILE cuda_src extra_headers ptx_generated extra_nvcc_args )
+function ( NVCC_COMPILE cuda_src extra_headers ptx_generated extra_nvcc_args extra_libs)
     get_filename_component ( cuda_src_we ${cuda_src} NAME_WE )
     get_filename_component ( cuda_src_dir ${cuda_src} DIRECTORY )
     set (cuda_ptx "${CMAKE_CURRENT_BINARY_DIR}/${cuda_src_we}.ptx" )
@@ -35,6 +35,14 @@ function ( NVCC_COMPILE cuda_src extra_headers ptx_generated extra_nvcc_args )
         set (NVCC_FTZ_FLAG "--ftz=true")
     endif ()
 
+    set (EXTRA_INCLUDES "")
+    foreach (LIB ${extra_libs})
+        get_target_property(INCLUDES ${LIB} INTERFACE_INCLUDE_DIRECTORIES)
+        foreach (INCLUDE_DIR ${INCLUDES})
+            list (APPEND EXTRA_INCLUDES "-I${INCLUDE_DIR}")
+        endforeach()
+    endforeach()
+    
     add_custom_command ( OUTPUT ${cuda_ptx}
         COMMAND ${CUDA_NVCC_EXECUTABLE}
             "-I${OPTIX_INCLUDES}"
@@ -43,6 +51,7 @@ function ( NVCC_COMPILE cuda_src extra_headers ptx_generated extra_nvcc_args )
             "-I${CMAKE_BINARY_DIR}/include"
             "-I${PROJECT_SOURCE_DIR}/src/include"
             "-I${PROJECT_SOURCE_DIR}/src/cuda_common"
+            ${EXTRA_INCLUDES}
             ${ALL_OpenImageIO_INCLUDES}
             ${ALL_IMATH_INCLUDES}
             "-DFMT_DEPRECATED=\"\""
@@ -56,7 +65,7 @@ function ( NVCC_COMPILE cuda_src extra_headers ptx_generated extra_nvcc_args )
             ${OSL_EXTRA_NVCC_ARGS}
             ${cuda_src} -o ${cuda_ptx}
         MAIN_DEPENDENCY ${cuda_src}
-        DEPENDS ${cuda_src} ${cuda_headers} oslexec
+        DEPENDS ${cuda_src} ${cuda_headers} ${extra_libs} oslexec
         WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}" )
 endfunction ()
 

src/cmake/testing.cmake

@@ -372,6 +372,7 @@ macro (osl_add_all_tests)
                 render-mx-layer
                 render-mx-sheen
                 render-microfacet render-oren-nayar
+                render-spi-thinlayer
                 render-uv render-veachmis render-ward
                 render-raytypes
                 select select-reg shaderglobals shortcircuit

src/libbsdl/CMakeLists.txt

@@ -0,0 +1,10 @@
+# Copyright Contributors to the Open Shading Language project.
+# SPDX-License-Identifier: BSD-3-Clause
+# https://github.com/AcademySoftwareFoundation/OpenShadingLanguage
+
+include(bsdl.cmake)
+
+# BSDL has small tables for doing spectral render in sRGB
+# but for other color spaces tell this function which ones
+# and it will bake Jakob-Hanika coefficient tables.
+add_bsdl_library(BSDL) # SPECTRAL_COLOR_SPACES "ACEScg")

src/libbsdl/README.md

@@ -0,0 +1,94 @@
+# BSDL library
+
+BSDL is the working title (Bidirectional Scattering Distribution Library) for
+this header only collection of BSDFs. It is self-contained depending only on
+Imath and intended to be used in any renderer. This is a build only dependency.
+
+The basic idea is that you choose a BSDF from it and give it a thin wrapper to
+integrate in your renderer. There is an example of this in OSL's testrender. The
+key features are:
+  * BSDFs provide constructor, eval and sample methods.
+  * Everything is inlined (split among _decl.h and _imple.h headers) to be GPU friendly.
+  * There is a template based autiomatic switch/case virtual dispatch mechanism to be GPU firendly.
+  * It works both in RGB or spectral mode (via hero wavelength).
+  * It includes the Sony Pictures Imageworks set of BSDFs used for movie production.
+
+## Pseudo virtual methods
+
+The header static_virtual.h provides the ```StaticVirtual<type1, type2, ...>```
+template. If you inherit from it will implement a dispatch() method to execute
+"virtual" methods covering ```type1, type2, ...``` using switch case statements.
+The idea was borrowed from PBRT but the implementation is different. See the
+use in testrender (shading.h/cpp).
+
+## The ```Power``` type
+
+To support both RGB and spectral render with the same code we use the 
+```Power``` type, which is just a float array. It has from RGB construction and
+conversion. But when 'lambda_0', the hero wavelength is zero, these are no-ops
+and pure RGB rendering is assueme. Take into account spectral support in BSDL
+is in very early stages.
+
+## Lookup tables
+
+A bunch of BSDFs use albedo lookup tables. Those are generated at build time
+very quickly and put in headers. You shouldn't have to do anything.
+
+## Usage
+
+Either ```include(bsdl.cmake)``` and call ```add_bsdl_library(my_name)``` to
+create a 'my_name' INTERFACE library that you then link, or 
+```add_subdirectory (path_to_libbsdl)``` to get it as 'BSDL'.
+
+For the integration, a config header needs to be defined as you can see in
+testrender's 'bsdl_config.h'.
+```cpp
+#define BSDL_INLINE        static inline OSL_HOSTDEVICE
+#define BSDL_INLINE_METHOD inline OSL_HOSTDEVICE
+#define BSDL_DECL          OSL_DEVICE
+#define BSDL_UNROLL()      // Do nothing
+
+#include <BSDL/config.h>
+
+#include <OpenImageIO/fmath.h>
+
+struct BSDLConfig : public bsdl::BSDLDefaultConfig {
+
+    // testrender won't do spectral render, just 3 channels covers RGB
+    static constexpr int HERO_WAVELENGTH_CHANNELS = 3;
+
+    struct Fast {
+        static BSDL_INLINE_METHOD float cosf(float x) { return OIIO::fast_cos(x); }
+        static BSDL_INLINE_METHOD float sinf(float x) { return OIIO::fast_sin(x); }
+        static BSDL_INLINE_METHOD float asinf(float x) { return OIIO::fast_asin(x); }
+        static BSDL_INLINE_METHOD float acosf(float x) { return OIIO::fast_acos(x); }
+        static BSDL_INLINE_METHOD float atan2f(float y, float x) { return OIIO::fast_atan2(y, x); }
+        static BSDL_INLINE_METHOD void sincosf(float x, float* s, float* c) { return OIIO::fast_sincos(x, s, c); }
+        static BSDL_INLINE_METHOD float sinpif(float x) { return OIIO::fast_sinpi(x); }
+        static BSDL_INLINE_METHOD float cospif(float x) { return OIIO::fast_cospi(x); }
+        static BSDL_INLINE_METHOD float expf(float x) { return OIIO::fast_exp(x); }
+        static BSDL_INLINE_METHOD float exp2f(float x) { return OIIO::fast_exp2(x); }
+        static BSDL_INLINE_METHOD float logf(float x) { return OIIO::fast_log(x); }
+        static BSDL_INLINE_METHOD float log2f(float x) { return OIIO::fast_log2(x); }
+        static BSDL_INLINE_METHOD float log1pf(float x) { return OIIO::fast_log1p(x); }
+        static BSDL_INLINE_METHOD float powf(float x, float y) { return OIIO::fast_safe_pow(x, y); }
+    };
+
+    // Don't care for colorspaces/spectral
+    static BSDL_INLINE_METHOD ColorSpaceTag current_color_space() { return ColorSpaceTag::sRGB; }
+    static BSDL_INLINE_METHOD const JakobHanikaLut* get_jakobhanika_lut(ColorSpaceTag cs) { return nullptr; }
+};
+```
+
+And this header should be included before you include anything else from BSDL.
+
+If spectral rendering is desired there is ready to use sRGB upsampling via
+spectral primaries by Mallett and Yuksel. For wide gamut color spaces like
+ACEScg we use Jakob and Hanika approach, but you have to ask for the tables
+in .cpp for from cmake:
+```
+add_bsdl_library(BSDL SPECTRAL_COLOR_SPACES "ACEScg" "ACES2065")
+``` 
+
+which will bake tables to two cpp files and return them in 'BSDL_LUTS_CPP'.
+Then you need to include that in your sources.

src/libbsdl/bsdl.cmake

@@ -0,0 +1,44 @@
+# Copyright Contributors to the Open Shading Language project.
+# SPDX-License-Identifier: BSD-3-Clause
+# https://github.com/AcademySoftwareFoundation/OpenShadingLanguage
+
+find_package(Threads REQUIRED)
+
+function(ADD_BSDL_LIBRARY NAME)
+    cmake_parse_arguments(PARSE_ARGV 1 bsdl "" "SUBDIR" "SPECTRAL_COLOR_SPACES")
+    # Bootstrap version of BSDL (without luts)
+    add_library(BSDL_BOOTSTRAP INTERFACE)
+    target_include_directories(BSDL_BOOTSTRAP INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/${bsdl_SUBDIR}/include)
+    target_link_libraries(BSDL_BOOTSTRAP INTERFACE ${ARNOLD_IMATH_TARGETS})
+
+    # LUT generation tool
+    set(BSDL_GEN_HEADERS ${CMAKE_CURRENT_BINARY_DIR}/${bsdl_SUBDIR}/geninclude)
+    add_executable (genluts ${CMAKE_CURRENT_SOURCE_DIR}/${bsdl_SUBDIR}/src/genluts.cpp)
+    target_link_libraries(genluts PRIVATE BSDL_BOOTSTRAP Threads::Threads)
+    file(MAKE_DIRECTORY ${BSDL_GEN_HEADERS}/BSDL/SPI)
+    add_custom_command(TARGET genluts POST_BUILD USES_TERMINAL COMMAND $<TARGET_FILE:genluts> ${BSDL_GEN_HEADERS}/BSDL/SPI
+                    COMMENT "Generating BSDL lookup tables ...")
+
+    if (DEFINED bsdl_SPECTRAL_COLOR_SPACES)
+        add_executable(jakobhanika_luts ${CMAKE_CURRENT_SOURCE_DIR}/${bsdl_SUBDIR}/src/jakobhanika_luts.cpp)
+        target_link_libraries(genluts PRIVATE Threads::Threads)
+        foreach(CS ${bsdl_SPECTRAL_COLOR_SPACES})
+            set(JACOBHANIKA_${CS} ${CMAKE_CURRENT_BINARY_DIR}/jakobhanika_${CS}.cpp)
+            list(APPEND BSDL_LUTS_CPP ${JACOBHANIKA_${CS}})
+            add_custom_command(
+                OUTPUT ${JACOBHANIKA_${CS}}
+                USES_TERMINAL
+                COMMAND $<TARGET_FILE:jakobhanika_luts> 64 ${JACOBHANIKA_${CS}} ${CS}
+                DEPENDS jakobhanika_luts
+                COMMENT "Generating Jakob-Hanika RGB-Spectrum ${CS} tables")
+        endforeach()
+        set(${NAME}_LUTS_CPP ${BSDL_LUTS_CPP} PARENT_SCOPE)
+    endif()
+
+    # Final BSDL library (with luts)
+    add_library(${NAME} INTERFACE)
+    target_include_directories(${NAME} INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/${bsdl_SUBDIR}/include)
+    target_link_libraries(${NAME} INTERFACE Imath::Imath)
+    target_include_directories(${NAME} INTERFACE ${BSDL_GEN_HEADERS})
+    add_dependencies(${NAME} genluts)
+endfunction()

src/libbsdl/include/BSDL/SPI/bsdf_backscatter_decl.h

@@ -0,0 +1,109 @@
+// Copyright Contributors to the Open Shading Language project.
+// SPDX-License-Identifier: BSD-3-Clause
+// https://github.com/AcademySoftwareFoundation/OpenShadingLanguage
+
+
+#pragma once
+
+#include <BSDL/bsdf_decl.h>
+
+BSDL_ENTER_NAMESPACE
+
+namespace spi {
+
+struct CharlieDist {
+    static constexpr float MIN_ROUGHNESS = 0.06f;
+
+    static BSDL_INLINE_METHOD float common_roughness(float alpha);
+    BSDL_INLINE_METHOD CharlieDist(float rough)
+        : a(CLAMP(rough, MIN_ROUGHNESS, 1.0f))
+    {
+    }
+
+    BSDL_INLINE_METHOD float D(const Imath::V3f& Hr) const;
+    BSDL_INLINE_METHOD float get_lambda(float cosNv) const;
+    BSDL_INLINE_METHOD float G2(const Imath::V3f& wo,
+                                const Imath::V3f& wi) const;
+    BSDL_INLINE_METHOD float roughness() const { return a; }
+
+private:
+    float a;
+};
+
+template<typename Dist> struct SheenMicrofacet {
+    // describe how tabulation should be done
+    static constexpr int Nc = 16;
+    static constexpr int Nr = 16;
+    static constexpr int Nf = 1;
+
+    static constexpr float get_cosine(int i)
+    {
+        return std::max(float(i) * (1.0f / (Nc - 1)), 1e-6f);
+    }
+    explicit BSDL_INLINE_METHOD SheenMicrofacet(float rough) : d(rough) {}
+    BSDL_INLINE_METHOD Sample eval(const Imath::V3f& wo,
+                                   const Imath::V3f& wi) const;
+    BSDL_INLINE_METHOD Sample sample(const Imath::V3f& wo, float randu,
+                                     float randv, float) const;
+    BSDL_INLINE_METHOD float roughness() const { return d.roughness(); }
+
+private:
+    Dist d;
+};
+
+struct CharlieSheen : public SheenMicrofacet<CharlieDist> {
+    explicit BSDL_INLINE_METHOD CharlieSheen(float, float rough, float)
+        : SheenMicrofacet<CharlieDist>(rough)
+    {
+    }
+    struct Energy {
+        float data[Nf * Nr * Nc];
+    };
+    static BSDL_INLINE_METHOD Energy& get_energy();
+
+    static const char* lut_header() { return "bsdf_backscatter_luts.h"; }
+    static const char* struct_name() { return "CharlieSheen"; }
+};
+
+template<typename BSDF_ROOT> struct CharlieLobe : public Lobe<BSDF_ROOT> {
+    using Base = Lobe<BSDF_ROOT>;
+    struct Data : public LayeredData {
+        Imath::V3f N;
+        Imath::C3f tint;
+        float roughness;
+        int doublesided;
+        using lobe_type = CharlieLobe;
+    };
+    template<typename D> static typename LobeRegistry<D>::Entry entry()
+    {
+        static_assert(
+            std::is_base_of<Data, D>::value);  // Make no other assumptions
+        using R = LobeRegistry<D>;
+        return { name(),
+                 { R::param(&D::closure), R::param(&D::N), R::param(&D::tint),
+                   R::param(&D::roughness),
+                   R::param(&D::doublesided, "doublesided"), R::close() } };
+    }
+
+    template<typename T>
+    BSDL_INLINE_METHOD CharlieLobe(T*, const BsdfGlobals& globals,
+                                   const Data& data);
+    static const char* name() { return "sheen"; }
+
+    BSDL_INLINE_METHOD Power albedo_impl() const { return Power(1 - Eo, 1); }
+
+    BSDL_INLINE_METHOD Sample eval_impl(const Imath::V3f& wo,
+                                        const Imath::V3f& wi) const;
+    BSDL_INLINE_METHOD Sample sample_impl(const Imath::V3f& wo,
+                                          const Imath::V3f& sample) const;
+
+private:
+    CharlieSheen sheen;
+    Power tint;
+    float Eo;
+    bool back;
+};
+
+}  // namespace spi
+
+BSDL_LEAVE_NAMESPACE