fix: refactor code and update init_params in debug_state

[Amnah199]

Related Issues

• fixes #9464
• remove the methods that are not needed any longer

Proposed Changes:

• remove remove_unserializable_data method (not needed after this PR is merged)
• Reintroduce init params in debug_state. This was removed for simplicity while converging with haystack/main.
• make breakpoint helper methods private

How did you test it?

Ran the tests

Notes for the reviewer

I would prefer to merge this PR before continuing with serialization on a separate one here. A single PR would be too jumbled.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[coveralls]

Pull Request Test Coverage Report for Build 15613105327

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 28 unchanged lines in 3 files lost coverage.
• Overall coverage decreased (-1.6%) to 87.208%

---

Files with Coverage Reduction	New Missed Lines	%
dataclasses/init.py	2	66.67%
core/pipeline/base.py	4	85.19%
core/pipeline/pipeline.py	22	63.29%

Totals
Change from base Build 15604645818:	-1.6%
Covered Lines:	1268
Relevant Lines:	1454

---

💛 - Coveralls

[Amnah199]

I would ignore the failing tests caused due to type mismatch of GeneratedAnswer in experimental and in core. This PR has been merged in haystack main which will resolves this.

[sjrl]

Same for adding all the Answer dataclasses. Is it just for having the new to_dict method in GeneratedAnswer?

[Amnah199]

Yes the new to_dict method in GeneratedAnswer serializes ChatMessage properly, and prevents JSON serializable error. I added the answer dataclasses and made AnswerBuilder here to prevent failing test cases but mismatch still exists.

[sjrl]

Yes the new to_dict method in GeneratedAnswer serializes ChatMessage properly, and prevents JSON serializable error. I added the answer dataclasses and made AnswerBuilder here to prevent failing test cases but mismatch still exists.

Okay I gotcha. Would it be possible then to only bring over GeneratedAnswer and not all of the other answer data classes if they aren’t needed? Also it’d be great if you could inherit from the haystack GeneratedAnswer and only add the code you have to. You can see how we did this for ChatMessage in this repo as well.

[Amnah199]

Yes the new to_dict method in GeneratedAnswer serializes ChatMessage properly, and prevents JSON serializable error. I added the answer dataclasses and made AnswerBuilder here to prevent failing test cases but mismatch still exists.

Okay I gotcha. Would it be possible then to only bring over GeneratedAnswer and not all of the other answer data classes if they aren’t needed? Also it’d be great if you could inherit from the haystack GeneratedAnswer and only add the code you have to. You can see how we did this for ChatMessage in this repo as well.

Right, I can do that. I initially added it as-is since I didn’t want to spend too much time patching GeneratedAnswer, given that it's going to be merged eventually. Just a heads-up — I might also need to add the AnswerJoiner to avoid the failing tests caused by mismatches.

[sjrl]

@Amnah199 it looks like the errors are Connection errors from Pipeline. We have an option in pipeline’s init that lets your turn off the connection validation. So then these errors should go away.

[sjrl]

In the docstring could we mention what keys are expected in resume_state?

[sjrl]

Especially since we have State in Agent now I think we should think about a more specific name than state in the function name. I see in the docs we call it "pipeline state", but I believe we are really loading a dict with keys "input_data", "pipeline_breakpoint", "pipeline_state" so it feels a bit odd to call the whole thing pipeline_state if there is a key called pipeline_state inside.

Do you have an idea on some better namings we could use?

[Amnah199]

We can stick to calling it debug_state or we consistently us resume_state internally so we can also keep that. WDYT?

[sjrl]

Maybe something different altogether like pipeline_snapshot?

[Amnah199]

Hmm if we opt for this then we should consistently use pipeline_snapshot instead of state everywhere. Including the resume_state param for run method

[sjrl]

I think this could make sense. I believe pipeline_snapshot is more informative than resume_state which from it's name sounds like it should be a boolean

[sjrl]

Suggested change

	Makes sure the breakpoint contains a valid components registered in the pipeline.
	Makes sure the breakpoint contains a valid component registered in the pipeline.

[sjrl]

One small comment, otherwise looks good!