Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[js/webgpu] allow to specify callback for profiling data #17820

Closed
wants to merge 5 commits into from
Closed

[js/webgpu] allow to specify callback for profiling data #17820

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
02600b0
[js/webgpu] allow to specify callback for profiling data
fs-eire Oct 6, 2023
657da70
Merge remote-tracking branch 'origin/main' into fs-eire/profiling-config
fs-eire Oct 12, 2023
08c0f23
revise program-manager.ts
fs-eire Oct 12, 2023
c79f072
Merge remote-tracking branch 'origin/main' into fs-eire/profiling-config
fs-eire Dec 6, 2023
4d4f664
update to latest
fs-eire Dec 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions js/common/lib/env.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,48 @@ export declare namespace Env {
async?: boolean;
}

export interface WebGpuProfilingDataV1TensorMetadata {
dims: readonly number[];
dataType: string;
}
export interface WebGpuProfilingDataV1 {
version: 1;
fs-eire marked this conversation as resolved.
Show resolved Hide resolved
inputsMetadata: readonly WebGpuProfilingDataV1TensorMetadata[];
outputsMetadata: readonly WebGpuProfilingDataV1TensorMetadata[];
kernelId: number;
kernelType: string;
kernelName: string;
startTime: number;
endTime: number;
}

export type WebGpuProfilingData = WebGpuProfilingDataV1;

export interface WebGpuFlags {
/**
* Set or get the profiling mode.
*
* @deprecated Use `env.webgpu.profiling.mode` instead. If `env.webgpu.profiling.mode` is set, this property will be
* ignored.
*/
profilingMode?: 'off'|'default';
/**
* Set or get the profiling configuration.
*/
profiling?: {
/**
* Set or get the profiling mode.
*
* @defaultValue `'off'`
*/
mode?: 'off'|'default';

/**
* Set or get a callback function when a profiling data is received. If not set, the profiling data will be
* printed to console.
*/
ondata?: (data: WebGpuProfilingData) => void;
};
/**
* Get the device for WebGPU.
*
Expand Down
6 changes: 4 additions & 2 deletions js/web/lib/wasm/jsep/init.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,8 +176,10 @@ export const init = async(module: OrtWasmModule, env: Env): Promise<void> => {
// jsepCreateKernel
(name: string, kernel: number, attribute: unknown) => backend.createKernel(
name, kernel, attribute,
env.debug || env.webgpu.profilingMode === 'default' ? module.UTF8ToString(module._JsepGetNodeName(kernel)) :
`${kernel}`),
env.debug || env.webgpu.profiling?.mode === 'default' ||
(!env.webgpu.profiling?.mode && env.webgpu.profilingMode === 'default') ?
module.UTF8ToString(module._JsepGetNodeName(kernel)) :
`${kernel}`),

// jsepReleaseKernel
(kernel: number) => backend.releaseKernel(kernel),
Expand Down
43 changes: 31 additions & 12 deletions js/web/lib/wasm/jsep/webgpu/program-manager.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,10 @@ export class ProgramManager {
dispatchGroup: [number, number, number]): void {
const device = this.backend.device;
const computePassEncoder = this.backend.getComputePassEncoder();
const profilingEnabled = this.backend.supportTimestampQuery && this.backend.env.webgpu.profilingMode === 'default';
const webgpuEnv = this.backend.env.webgpu;
const profilingEnabled = this.backend.supportTimestampQuery &&
(webgpuEnv.profiling?.mode === 'default' ||
(!webgpuEnv.profiling?.mode && webgpuEnv.profilingMode === 'default'));
if (profilingEnabled) {
// profiling write start timestamp

Expand Down Expand Up @@ -103,17 +106,33 @@ export class ProgramManager {
}

this.backend.gpuDataManager.release(syncData.id);
let inputShapes = '';
inputsTensorView.forEach((value, i) => {
inputShapes += `input[${i}]: [${value.dims}] | ${tensorDataTypeEnumToString(value.dataType)}, `;
});
let outputShapes = '';
buildArtifact.programInfo.outputs.forEach((value, i) => {
outputShapes += `output[${i}]: [${value.dims}] | ${tensorDataTypeEnumToString(value.dataType)}, `;
});
// eslint-disable-next-line no-console
console.log(`[profiling] kernel "${kernelId}|${kernelName}" ${inputShapes}${outputShapes}execution time: ${
endTime - startTime} ns`);
if (this.backend.env.webgpu.profiling?.ondata) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default startTime is always 0. And any use case for profilingTimeBase?

       if (typeof this.backend.profilingTimeBase === 'undefined') {
          this.backend.profilingTimeBase = startTimeU64;
        }
        const startTime = Number(startTimeU64 - this.backend.profilingTimeBase);
        const endTime = Number(endTimeU64 - this.backend.profilingTimeBase);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backend.profilingTimeBase is designed to log the time of the start time of the first completed kernel (startTime64). By using this, the startTime and endTime can be relatively small to fit into int53.

this.backend.env.webgpu.profiling.ondata({
version: 1,
inputsMetadata: inputsTensorView.map(
value => ({dims: value.dims, dataType: tensorDataTypeEnumToString(value.dataType)})),
outputsMetadata: buildArtifact.programInfo.outputs.map(
value => ({dims: value.dims, dataType: tensorDataTypeEnumToString(value.dataType)})),
kernelId,
kernelType: kernelInfo[0],
kernelName: kernelInfo[1],
startTime,
endTime,
});
} else {
// if no callback is provided, print the profiling message to console
let inputShapes = '';
fs-eire marked this conversation as resolved.
Show resolved Hide resolved
inputsTensorView.forEach((value, i) => {
inputShapes += `input[${i}]: [${value.dims}] | ${tensorDataTypeEnumToString(value.dataType)}, `;
});
let outputShapes = '';
buildArtifact.programInfo.outputs.forEach((value, i) => {
outputShapes += `output[${i}]: [${value.dims}] | ${tensorDataTypeEnumToString(value.dataType)}, `;
});
// eslint-disable-next-line no-console
console.log(`[profiling] kernel "${kernelId}|${kernelName}" ${inputShapes}${outputShapes}execution time: ${
endTime - startTime} ns`);
}
});
}

Expand Down
2 changes: 1 addition & 1 deletion js/web/test/test-main.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ if (options.globalEnvFlags) {
ort.env.wasm.initTimeout = flags.wasm.initTimeout;
}
if (flags.webgpu?.profilingMode !== undefined) {
ort.env.webgpu.profilingMode = flags.webgpu.profilingMode;
ort.env.webgpu.profiling = {mode: flags.webgpu.profilingMode};
}
if (flags.webgpu?.validateInputContent !== undefined) {
ort.env.webgpu.validateInputContent = flags.webgpu.validateInputContent;
Expand Down