Skip to content

Commit

Permalink
v0.7.6.2 Release
Browse files Browse the repository at this point in the history
v0.7.6.2 Release
  • Loading branch information
jdsouza90 committed Sep 13, 2021
1 parent c4154fa commit 3bd2be6
Show file tree
Hide file tree
Showing 27 changed files with 521 additions and 258 deletions.
6 changes: 6 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Changelog (v0.7.6.2)
- 3D Body Pose Keypoint Tracking
- Quality improvement in accuracy and stability
- Property NvAR_Parameter_Config(NVAR_MODE) is now NvAR_Parameter_Config(Mode)
- Migrated to TensorRT 8.0.1.6
- Migrated to CUDA 11.3u1
22 changes: 11 additions & 11 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.12.0)
cmake_minimum_required(VERSION 3.10.0)

# Set path where samples will be installed
set(CMAKE_INSTALL_PREFIX ${CMAKE_SOURCE_DIR} CACHE PATH "Path to where the samples will be installed")
Expand All @@ -8,16 +8,13 @@ project(NvAR_SDK CXX)
set(CMAKE_CONFIGURATION_TYPES "Release")

# Require C++11 and disable non-standard extensions

set(CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake" ${CMAKE_MODULE_PATH})

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

add_definitions(-DNOMINMAX -DWIN32_LEAN_AND_MEAN)

# Set common build path for all targets
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})

set(SDK_INCLUDES_PATH ${CMAKE_CURRENT_SOURCE_DIR}/nvar/include)

# Add target for nvARPose
Expand All @@ -27,10 +24,13 @@ target_include_directories(nvARPose INTERFACE ${SDK_INCLUDES_PATH})
# Add target for NVCVImage
add_library(NVCVImage INTERFACE)
target_include_directories(NVCVImage INTERFACE ${SDK_INCLUDES_PATH})
if(UNIX)
target_link_libraries(NVCVImage INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/bin/libNVCVImage.so)
endif(UNIX)

add_definitions(-DNOMINMAX -DWIN32_LEAN_AND_MEAN)

# Set common build path for all targets
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})


set(ENABLE_SAMPLES TRUE)
add_subdirectory(samples)

26 changes: 20 additions & 6 deletions README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The SDK has the following features:
<img src="https://github.com/NVIDIA/MAXINE-AR-SDK/blob/master/resources/ar_004.png" alt="Face 3D mesh and tracking" width="320" height="180"/>
</p>
</p><p align="center">
<img src="https://github.com/NVIDIA/MAXINE-AR-SDK/blob/master/resources/ar_005.png" alt="3D Body Pose and tracking" width="320" height="180"/>
<img src="https://github.com/NVIDIA/MAXINE-AR-SDK/blob/master/resources/ar_005.png" alt="Body 3D Pose and tracking" width="480" height="270"/>
</p>

The SDK provides two sample applications that demonstrate the features listed above in real time by using a webcam or offline videos.
Expand All @@ -28,17 +28,19 @@ The SDK provides two sample applications that demonstrate the features listed ab
NVIDIA MAXINE AR SDK is distributed in the following parts:

- This open source repository that includes the [SDK API and proxy linking source code](https://github.com/NVIDIA/MAXINE-AR-SDK/tree/master/nvar), and [sample applications and their dependency libraries](https://github.com/NVIDIA/MAXINE-AR-SDK/tree/master/samples).
- An installer hosted on [NVIDIA Maxine developer page](https://www.nvidia.com/broadcast-sdk-resources) that installs the SDK DLLs, the models, and the SDK dependency libraries.
- An installer hosted on [NVIDIA Maxine End-user Redistributables page](https://www.nvidia.com/broadcast-sdk-resources) that installs the SDK DLLs, the models, and the SDK dependency libraries.

Please refer to [SDK programming guide](https://github.com/NVIDIA/MAXINE-AR-SDK/blob/master/docs/NVIDIA%20AR%20SDK%20Programming%20Guide.pdf) for configuring and integrating the SDK, compiling and running the sample applications. Please visit the [NVIDIA MAXINE AR SDK](https://developer.nvidia.com/maxine-getting-started) webpage for more information about the SDK.
Please refer to [SDK System guide](https://docs.nvidia.com/deeplearning/maxine/ar-sdk-system-guide/index.html) for configuring and integrating the SDK, compiling and running the sample applications. Please visit the [NVIDIA MAXINE AR SDK](https://developer.nvidia.com/maxine-getting-started) webpage for more information about the SDK.

## System requirements
The SDK is supported on NVIDIA GPUs that are based on the NVIDIA® Turing™ or Ampere™ architecture and have Tensor Cores.

* Windows OS supported: 64-bit Windows 10
* Microsoft Visual Studio: 2015 (MSVC14.0) or later
* Windows OS supported: 64-bit Windows 10 or later
* Microsoft Visual Studio: 2017 (MSVC15.0) or later
* CMake: v3.12 or later
* NVIDIA Graphics Driver for Windows: 455.57 or later
* NVIDIA Graphics Driver for Windows: 465.89 or later
* NVIDIA CUDA Toolkit: 11.3.1
* NVIDIA TensorRT: 8.0.1.6

## NVIDIA MAXINE Branding Guidelines
If you integrate an NVIDIA MAXINE SDK within your product, please follow the required branding guidelines that are available [here](
Expand Down Expand Up @@ -66,3 +68,15 @@ The open source repository includes the source code to build the sample applicat
* In CMake, to open Visual Studio, click Open Project.
* In Visual Studio, select Build > Build Solution.

## Documentation

Please refer to the online documentation guides -
* [NVIDIA AR SDK Programming Guide](https://docs.nvidia.com/deeplearning/maxine/ar-sdk-programming-guide/index.html)
* [NVIDIA AR SDK System Guide](https://docs.nvidia.com/deeplearning/maxine/ar-sdk-system-guide/index.html)
* [NvCVImage API Guide](https://docs.nvidia.com/deeplearning/maxine/nvcvimage-api-guide/index.html)

PDF versions of these guides are also available at the following locations -
* [NVIDIA AR SDK Programming Guide](https://docs.nvidia.com/deeplearning/maxine/pdf/ar-sdk-programming-guide.pdf)
* [NVIDIA AR SDK System Guide](https://docs.nvidia.com/deeplearning/maxine/pdf/ar-sdk-system-guide.pdf)
* [NvCVImage API Guide](https://docs.nvidia.com/deeplearning/maxine/pdf/nvcvimage-api-guide.pdf)

Binary file removed docs/NVIDIA AR SDK Programming Guide.pdf
Binary file not shown.
43 changes: 41 additions & 2 deletions nvar/include/nvAR_defs.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@
#define NvAR_API
#endif // OS dependencies

// TODO: Change the representation to x,y,z instead of array
typedef struct NvAR_Vector3f
{
float vec[3];
Expand Down Expand Up @@ -96,7 +95,8 @@ typedef struct NvAR_RenderingParams {

// Parameters provided by client application
typedef const char* NvAR_FeatureID;
#define NvAR_Feature_FaceDetection "FaceDetection"
#define NvAR_Feature_FaceBoxDetection "FaceBoxDetection"
#define NvAR_Feature_FaceDetection "FaceDetection" // deprecated in favor of FaceBox
#define NvAR_Feature_LandmarkDetection "LandmarkDetection"
#define NvAR_Feature_Face3DReconstruction "Face3DReconstruction"
#define NvAR_Feature_BodyDetection "BodyDetection"
Expand Down Expand Up @@ -174,6 +174,45 @@ NvAR_Parameter_Output(Pose) - OPTIONAL
NvAR_Parameter_Output(LandmarksConfidence) - OPTIONAL
NvAR_Parameter_Output(ExpressionCoefficients) - OPTIONAL
NvAR_Parameter_Output(ShapeEigenValues) - OPTIONAL
*******NvAR_Feature_BodyDetection*******
Config:
NvAR_Parameter_Config(FeatureDescription)
NvAR_Parameter_Config(CUDAStream)
NvAR_Parameter_Config(TRTModelDir)
NvAR_Parameter_Config(Temporal)
Input:
NvAR_Parameter_Input(Image)
Output:
NvAR_Parameter_Output(BoundingBoxes)
NvAR_Parameter_Output(BoundingBoxesConfidence) - OPTIONAL
*******NvAR_Feature_BodyPoseEstimation*******
Config:
NvAR_Parameter_Config(FeatureDescription)
NvAR_Parameter_Config(CUDAStream)
NvAR_Parameter_Config(ModelDir)
NvAR_Parameter_Config(BatchSize)
NvAR_Parameter_Config(Mode)
NvAR_Parameter_Config(NumKeyPoints)
NvAR_Parameter_Config(ReferencePose)
NvAR_Parameter_Config(Temporal)
NvAR_Parameter_Config(UseCudaGraph)
NvAR_Parameter_Config(FocalLength)
Input:
NvAR_Parameter_Input(Image)
NvAR_Parameter_Input(BoundingBoxes) - OPTIONAL
Output:
NvAR_Parameter_Output(BoundingBoxes) - OPTIONAL
NvAR_Parameter_Output(BoundingBoxesConfidence) - OPTIONAL
NvAR_Parameter_Output(KeyPoints)
NvAR_Parameter_Output(KeyPoints3D)
NvAR_Parameter_Output(JointAngles)
NvAR_Parameter_Output(KeyPointsConfidence) - OPTIONAL
*/


Expand Down
59 changes: 42 additions & 17 deletions nvar/include/nvCVImage.h
Original file line number Diff line number Diff line change
Expand Up @@ -204,21 +204,24 @@ NvCVImage {
//! \param[in] dstY The top coordinate of the dst rectangle.
//! \param[in] width The width of the rectangle to be copied, in pixels.
//! \param[in] height The height of the rectangle to be copied, in pixels.
//! \param[in] stream the CUDA stream.
//! \note NvCVImage_Transfer() can handle more cases.
//! \return NVCV_SUCCESS if successful
//! \return NVCV_ERR_MISMATCH if the formats are different
//! \return NVCV_ERR_CUDA if a CUDA error occurred
//! \return NVCV_ERR_PIXELFORMAT if the pixel format is not yet accommodated.
inline NvCV_Status copyFrom(const NvCVImage *src, int srcX, int srcY, int dstX, int dstY, unsigned width, unsigned height);
inline NvCV_Status copyFrom(const NvCVImage *src, int srcX, int srcY, int dstX, int dstY,
unsigned width, unsigned height, struct CUstream_st* stream = 0);

//! Copy from one image to another. This works for CPU->CPU, CPU->GPU, GPU->GPU, and GPU->CPU.
//! \param[in] src The source image from which to copy.
//! \param[in] stream the CUDA stream.
//! \note NvCVImage_Transfer() can handle more cases.
//! \return NVCV_SUCCESS if successful
//! \return NVCV_ERR_MISMATCH if the formats are different
//! \return NVCV_ERR_CUDA if a CUDA error occurred
//! \return NVCV_ERR_PIXELFORMAT if the pixel format is not yet accommodated.
inline NvCV_Status copyFrom(const NvCVImage *src);
inline NvCV_Status copyFrom(const NvCVImage *src, struct CUstream_st* stream = 0);

#endif // ___cplusplus
} NvCVImage;
Expand Down Expand Up @@ -466,6 +469,8 @@ NvCV_Status NvCV_API NvCVImage_TransferRect(
//! \param[in] tmp a staging image.
//! \return NVCV_SUCCESS if the operation was completed successfully.
//! \note The actual transfer region may be smaller, because the rects are clipped against the images.
//! \note This is supplied for use with YUV buffers that do not have the standard structure
//! that are expected for NvCVImage_Transfer() and NvCVImage_TransferRect.
NvCV_Status NvCV_API NvCVImage_TransferFromYUV(
const void *y, int yPixBytes, int yPitch,
const void *u, const void *v, int uvPixBytes, int uvPitch,
Expand All @@ -492,6 +497,8 @@ NvCV_Status NvCV_API NvCVImage_TransferFromYUV(
//! \param[in] tmp a staging image.
//! \return NVCV_SUCCESS if the operation was completed successfully.
//! \note The actual transfer region may be smaller, because the rects are clipped against the images.
//! \note This is supplied for use with YUV buffers that do not have the standard structure
//! that are expected for NvCVImage_Transfer() and NvCVImage_TransferRect.
NvCV_Status NvCV_API NvCVImage_TransferToYUV(
const NvCVImage *src, const NvCVRect2i *srcRect,
const void *y, int yPixBytes, int yPitch,
Expand All @@ -507,7 +514,9 @@ NvCV_Status NvCV_API NvCVImage_TransferToYUV(
//! \param[in,out] im the image to be mapped.
//! \param[in] stream the stream on which the mapping is to be performed.
//! \return NVCV_SUCCESS is the operation was completed successfully.
NvCV_Status NvCV_API NvCVImage_MapResource(NvCVImage *im, struct CUstream_st *stream);
//! \note This is an experimental API. If you find it useful, please respond to [email protected],
//! otherwise we may drop support.
/* EXPERIMENTAL */ NvCV_Status NvCV_API NvCVImage_MapResource(NvCVImage *im, struct CUstream_st *stream);


//! After transfer by CUDA, the texture resource must be unmapped in order to be used by the graphics system again.
Expand All @@ -516,7 +525,9 @@ NvCV_Status NvCV_API NvCVImage_MapResource(NvCVImage *im, struct CUstream_st *st
//! \param[in,out] im the image to be mapped.
//! \param[in] stream the CUDA stream on which the mapping is to be performed.
//! \return NVCV_SUCCESS is the operation was completed successfully.
NvCV_Status NvCV_API NvCVImage_UnmapResource(NvCVImage *im, struct CUstream_st *stream);
//! \note This is an experimental API. If you find it useful, please respond to [email protected],
//! otherwise we may drop support.
/* EXPERIMENTAL */ NvCV_Status NvCV_API NvCVImage_UnmapResource(NvCVImage *im, struct CUstream_st *stream);


//! Composite one source image over another using the given matte.
Expand All @@ -530,15 +541,17 @@ NvCV_Status NvCV_API NvCVImage_UnmapResource(NvCVImage *im, struct CUstream_st *
//! \return NVCV_ERR_PIXELFORMAT if the pixel format is not accommodated.
//! \return NVCV_ERR_MISMATCH if either the fg & bg & dst formats do not match, or if fg & bg & dst & mat are not
//! in the same address space (CPU or GPU).
//! \bug Though RGBA destinations are accommodated, the A channel is not updated at all.
#if RTX_CAMERA_IMAGE == 0
NvCV_Status NvCV_API NvCVImage_Composite(const NvCVImage *fg, const NvCVImage *bg, const NvCVImage *mat, NvCVImage *dst,
struct CUstream_st *stream);
#else // RTX_CAMERA_IMAGE == 1 // No GPU acceleration
NvCV_Status NvCV_API NvCVImage_Composite(const NvCVImage *fg, const NvCVImage *bg, const NvCVImage *mat, NvCVImage *dst);
#endif // RTX_CAMERA_IMAGE == 1


//! Composite one source image over another using the given matte.
//! Not all pixel format combinations are accommodated.
//! This accommodates all RGB and RGBA formats, with u8 and f32 components.
//! \param[in] fg the foreground source image.
//! \param[in] fgOrg the upper-left corner of the fg image to be composited (NULL implies (0,0)).
//! \param[in] bg the background source image.
Expand Down Expand Up @@ -566,17 +579,27 @@ NvCV_Status NvCV_API NvCVImage_CompositeRect(
NvCVImage *dst, const NvCVPoint2i *dstOrg,
struct CUstream_st *stream);

//! Composite a BGRu8 source image over a constant color field using the given matte.
//! \param[in] src the source BGRu8 (or RGBu8) image.
//! \param[in] mat the matte Yu8 (or Au8) image, indicating where the src should come through.
//! \param[in] bgColor the desired flat background color, with the same component ordering as the src and dst.
//! \param[in,out] dst the destination BGRu8 (or RGBu8) image. May be the same as src.

//! Composite a source image over a constant color field using the given matte.
//! \param[in] src the source image.
//! \param[in] mat the matte image, indicating where the src should come through.
//! \param[in] bgColor pointer to a location holding the desired flat background color, with the same format
//! and component ordering as the dst. This acts as a 1x1 background pixel buffer,
//! so should reside in the same memory space (CUDA or CPU) as the other buffers.
//! \param[in,out] dst the destination image. May be the same as src.
//! \return NVCV_SUCCESS if the operation was successful.
//! \return NVCV_ERR_PIXELFORMAT if the pixel format is not accommodated.
//! \bug This is only implemented for 3-component u8 src and dst, and 1-component mat,
//! where all images are resident on the CPU.
//! \return NVCV_ERR_MISMATCH if fg & mat & dst & bgColor are not in the same address space (CPU or GPU).
//! \note The bgColor must remain valid until complete; this is an important consideration especially if
//! the buffers are on the GPU and NvCVImage_CompositeOverConstant() runs asynchronously.
//! \bug Though RGBA destinations are accommodated, the A channel is not updated at all.
NvCV_Status NvCV_API NvCVImage_CompositeOverConstant(
const NvCVImage *src, const NvCVImage *mat, const unsigned char bgColor[3], NvCVImage *dst);
#if RTX_CAMERA_IMAGE == 0
const NvCVImage *src, const NvCVImage *mat, const void *bgColor, NvCVImage *dst, struct CUstream_st *stream
#else // RTX_CAMERA_IMAGE == 1
const NvCVImage *src, const NvCVImage *mat, const unsigned char bgColor[3], NvCVImage *dst
#endif // RTX_CAMERA_IMAGE
);


//! Flip the image vertically.
Expand Down Expand Up @@ -649,24 +672,26 @@ NvCVImage::~NvCVImage() { NvCVImage_Dealloc(this); }
********************************************************************************/

NvCV_Status NvCVImage::copyFrom(const NvCVImage *src, int srcX, int srcY, int dstX, int dstY, unsigned wd,
unsigned ht) {
unsigned ht, struct CUstream_st* stream) {
#if RTX_CAMERA_IMAGE // This only works for chunky images
NvCVImage srcView, dstView;
NvCVImage_InitView(&srcView, const_cast<NvCVImage *>(src), srcX, srcY, wd, ht);
NvCVImage_InitView(&dstView, this, dstX, dstY, wd, ht);
return NvCVImage_Transfer(&srcView, &dstView, 1.f, 0, nullptr);
return NvCVImage_Transfer(&srcView, &dstView, 1.f, stream, nullptr);
#else // !RTX_CAMERA_IMAGE bug fix for non-chunky images
NvCVRect2i srcRect = { (int)srcX, (int)srcY, (int)wd, (int)ht };
NvCVPoint2i dstPt = { (int)dstX, (int)dstY };
return NvCVImage_TransferRect(src, &srcRect, this, &dstPt, 1.f, 0, nullptr);
return NvCVImage_TransferRect(src, &srcRect, this, &dstPt, 1.f, stream, nullptr);
#endif // RTX_CAMERA_IMAGE
}

/********************************************************************************
* copy image
********************************************************************************/

NvCV_Status NvCVImage::copyFrom(const NvCVImage *src) { return NvCVImage_Transfer(src, this, 1.f, 0, nullptr); }
NvCV_Status NvCVImage::copyFrom(const NvCVImage *src, struct CUstream_st* stream) {
return NvCVImage_Transfer(src, this, 1.f, stream, nullptr);
}


#endif // ___cplusplus
Expand Down
Loading

0 comments on commit 3bd2be6

Please sign in to comment.