[Dev] Make PRs that doesn't match our PR format draft automatically

[kou]

Describe the enhancement requested

We have a CI job that checks our PR format:

arrow/.github/workflows/dev_pr.yml

Lines 63 to 72 in 6378639

	- name: Check title
	if: |
	(github.event.action == 'opened' ||
	github.event.action == 'edited')
	uses: actions/github-script@v9
	with:
	github-token: ${{ secrets.GITHUB_TOKEN }}
	script: |
	const script = require(`${process.env.GITHUB_WORKSPACE}/.github/workflows/dev_pr/title_check.js`);
	script({github, context});

If it detects format mismatch, it adds a comment to notify it: https://github.com/apache/arrow/blob/main/.github/workflows/dev_pr/title_check.md

How about making the PR draft automatically in addition of the comment? Maintainers can ignore these PRs until they match our PR format.

Component(s)

Developer Tools

[raulcd]

Sounds reasonable to me. I think the obvious offenders are the agent generated ones without human intervention, maybe the efforts from @thisisnic on adding new AGENTS.md files help with those.

[rok]

Agreed! We just want to give clear human readable feedback to not introduce an unnecessary friction point.

[thisisnic]

Comments welcomed on #50375 if there are any other common AI failures we can try to account for.

Do we also perhaps want to add an auto-comment for issues which have been created without our issue template, and lack a "Type" and "Component" label as there are AI-generated ones which look like this and so are less likely to get in front of the relevant person.

[kou]

@Reranko05 Do you want to work on this?

[Reranko05]

@kou Sure! I'll take a look.