Skip to content

media: i2c: imx500: input tensor injection#7040

Merged
roliver-rpi merged 1 commit intorpi-6.12.yfrom
dev/roliver/imx500_it_injection
Sep 9, 2025
Merged

media: i2c: imx500: input tensor injection#7040
roliver-rpi merged 1 commit intorpi-6.12.yfrom
dev/roliver/imx500_it_injection

Conversation

@roliver-rpi
Copy link
Copy Markdown
Contributor

Input tensor injection is a debug feature that allows a user-controlled input to be passed directly to IMX500's inference engine (bypassing the in-built ISP).

Three new custom controls are added to ENABLE_INJECTION before streaming begins, to provide appropriate input tensors via an INPUT_TENSOR_FD, and to provide notification of DNN results in the sensor output via INJECTION_CMP_FRM.

Copy link
Copy Markdown
Contributor

@pelwell pelwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have much to say about this, except that you could comply with checkpatch's implicit suggestion to remove those elses.

Copy link
Copy Markdown
Contributor

@6by9 6by9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Largely coding style nits.

Comment thread drivers/media/i2c/imx500.c Outdated
if (hndsk_reg.val == IMX500_DD_DAT_INJECTION_HNDSK_TRANS_COMP) {
break;
} else if (
hndsk_reg.val ==
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this up onto the same line as the if (

Comment thread drivers/media/i2c/imx500.c Outdated
/* Check if we've reached the final completion state */
if (hndsk_reg.val == IMX500_DD_DAT_INJECTION_HNDSK_TRANS_COMP) {
break;
} else if (
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As checkpatch and Phil have said, no need for the else on any of these as you've jumped out of the loop or returned in the error handling of all of them.

Comment thread drivers/media/i2c/imx500.c Outdated
ret = cci_write(imx500->regmap, IMX500_REG_STRM_MODE_SEL, 4, NULL);
if (ret) {
dev_err(&client->dev, "Failed to enable DNN processing\n");
return ret;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this return ret and braces, and just return ret; at line 2113.

Comment thread drivers/media/i2c/imx500.c Outdated
}

const u8 *tensor_data;
size_t tensor_data_size;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we get a warning for variables being defined in the middle of a function?

Comment thread drivers/media/i2c/imx500.c Outdated
break;
case V4L2_CID_USER_IMX500_INJECTION_CMP_FRM:
ctrl->val = imx500->current_injection_cmp_frm;
dev_info(&client->dev, "Injection comparison frame: %u\n",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be pretty chatty. dev_dbg?

Copy link
Copy Markdown
Contributor

@naushir naushir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from the styling things mentioned, I see no problems with the logic of the code and state machine itself.

Input tensor injection is a debug feature that allows a user-controlled
input to be passed directly to IMX500's inference engine (bypassing the
in-built ISP).

Three new custom controls are added to ENABLE_INJECTION before streaming
begins, to provide appropriate input tensors via an INPUT_TENSOR_FD, and
to provide notification of DNN results in the sensor output via
INJECTION_CMP_FRM.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>
@roliver-rpi roliver-rpi force-pushed the dev/roliver/imx500_it_injection branch from f4da5c4 to a016921 Compare September 9, 2025 12:22
@roliver-rpi roliver-rpi marked this pull request as ready for review September 9, 2025 12:38
@roliver-rpi
Copy link
Copy Markdown
Contributor Author

I believe all PR comments have been addressed. Are we happy to merge @6by9 ?

@pelwell
Copy link
Copy Markdown
Contributor

pelwell commented Sep 9, 2025

LGTM!

Copy link
Copy Markdown
Contributor

@6by9 6by9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy for it to be merged as is despite these comments.

imx500->input_tensor_fd =
v4l2_ctrl_new_custom(ctrl_hdlr, &input_tensor_fd, NULL);
imx500->injection_cmp_frm =
v4l2_ctrl_new_custom(ctrl_hdlr, &injection_cmp_frm, NULL);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 3 controls, but not increased the hint passed in ret = v4l2_ctrl_handler_init(ctrl_hdlr, 16); at line 3374.

I make it 17 controls total (two added by v4l2_ctrl_new_fwnode_properties), which implies that the number was wrong before.

static int __must_check imx500_spi_write(struct imx500 *state, const u8 *data,
size_t size)
{
if (size % 4 || size > ONE_MIB)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: size % 4 would be the same as size & 3. The compiler may well do that anyway.

return ret;
}

if (tensor_data_size % 4 != 0) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if (tensor_data_size & 3)

@roliver-rpi roliver-rpi merged commit 933bb6b into rpi-6.12.y Sep 9, 2025
14 of 26 checks passed
popcornmix added a commit to raspberrypi/firmware that referenced this pull request Sep 15, 2025
kernel: Enable interrupts on CM5 ethernet phy (again)
See: raspberrypi/linux#6963

kernel: media: i2c: imx500: input tensor injection
See: raspberrypi/linux#7040
popcornmix added a commit to raspberrypi/rpi-firmware that referenced this pull request Sep 15, 2025
kernel: Enable interrupts on CM5 ethernet phy (again)
See: raspberrypi/linux#6963

kernel: media: i2c: imx500: input tensor injection
See: raspberrypi/linux#7040
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants