Add fake generic and modify tests #10266

MozammilQ · 2023-06-12T17:41:09Z

Summary

(1/4) --> This is the first PR of the FakeBackends refactoring epic.

This PR adds a FakeGeneric(BackendV2) backend for qiskit unit testing purposes. It only updates the uses of V2 FakeBackends in test/python/compiler/test_transpiler.py. The rest of the unit tests using V2 fake backends are updated in #10918.

Details and comments

Follow-up to test generalized use in tests: Update remaining unit tests to use FakeGeneric #10918
Partially fixes Add generic Fake Backends and update use in Qiskit unit tests #10954.
V1 FakeBackends are handled in: Add generic V1 Fake Backends, replace use in tests #10952

Note: I can move the GenericTarget class to an independent file once the approach is agreed upon and I am still not 100% convinced by the FakeGeneric name for the backend.

…rking code

…d with FakeGeneric

…th instruction_dict in fake_generic.py; added support of grid type of coupling map; using lower number of qubits, passed all tests of test_transpiler except one method ( test_parallel_dispatch_lazy_cal_loading )

…t_fake_generic.py

qiskit-bot · 2023-06-12T17:41:14Z

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the the following people are requested to review this:

@Qiskit/terra-core

MozammilQ · 2023-06-12T17:43:25Z

@HuangJunye, I humbly request you to see if this is good enough :)

test/python/compiler/test_transpiler.py

HuangJunye · 2023-06-13T09:44:42Z

@HuangJunye, I humbly request you to see if this is good enough :)

Thanks for submitting the PR. I will review it.

This PR is part of QAMP Spring 23: qiskit-advocate/qamp-spring-23#26

atharva-satpute

Please fix the typos

qiskit/providers/fake_provider/fake_generic.py

test/python/providers/fake_provider/test_fake_generic.py

Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com>

MozammilQ · 2023-07-01T11:51:38Z

@atharva-satpute , done with the changes :)

HuangJunye · 2023-10-02T10:37:09Z

@MozammilQ Thanks for the awesome work here! Qiskit core developers really like it. Is it ok for them to take over the PR to finish it up?

MozammilQ · 2023-10-02T11:49:22Z

To Qiskit Core developers: Thanks for the kind consideration, this work is not possible without the kind guidance of my mentor @HuangJunye , credit to any such work goes to him ! :)

@HuangJunye , around the time we last talked, I did some further work and did some more commits, today I have opened a draft PR for that work.
Here it is:
#10942
#10943

MozammilQ · 2023-10-02T12:04:08Z

@HuangJunye , Yes, it is ok for Qiskit Core to take over the PR to finish it up!!

HuangJunye · 2023-10-02T13:23:06Z

To Qiskit Core developers: Thanks for the kind consideration, this work is not possible without the kind guidance of my mentor @HuangJunye , credit to any such work goes to him ! :)

@HuangJunye , around the time we last talked, I did some further work and did some more commits, today I have opened a draft PR for that work. Here it is: #10942 #10943

Great. Thank you for adding the new work as well. The core team will take a look and work with them too.

ElePT

Thank you very much @MozammilQ for your contribution. I have added a couple of commits to try to get the PR up to speed, and will now add some comments for reference of the dev. team (no need to act on them) @HuangJunye.

qiskit/providers/fake_provider/fake_generic.py

ElePT · 2024-01-27T13:49:11Z

Thanks for the feedback, I have addressed all of the review comments in 54496db, except for the backend name. I think that the candidates are:

ConfigurableFakeBackend or ConfigurableFakeBackendV2
GenericBackend or GenericBackendV2
GenericFakeBackend or GenericFakeBackendV2

I am fine settling for GenericBackendV2, if that seems like a better option.

ElePT · 2024-01-29T10:24:25Z

I've renamed it to GenericBackendV2 in a36acaf. The module docs are still improvable, but I was planning to update them in #11376 once the deprecated classes are out, and include the examples that are currently in the release note.

kevinhartman

Looking good, a few minor comments / typos.

qiskit/providers/fake_provider/fake_generic.py

kevinhartman · 2024-01-29T16:46:33Z

qiskit/providers/fake_provider/fake_generic.py

+]
+
+
+class GenericFakeBackend(BackendV2):


I guess the way I see it is, if you don't need a simulator, subclassing this thing won't do anything for you, in which case you really ought to define your own subclass. The name GenericBackendV2 is fine assuming the term "generic" implies "simulator-based", I'm just less convinced that this is a minimal implementor of BackendV2.

kevinhartman · 2024-01-29T16:50:16Z

qiskit/providers/fake_provider/fake_generic.py

+            basis_gates if basis_gates is not None else ["cx", "id", "rz", "sx", "x"]
+        )
+        for name in ["reset", "delay", "measure"]:
+            if name not in self._basis_gates:


One trick I found when working in my branch was to use Target.operation_names to test for instruction presence in constant time. That's backed by a dictionary, I believe, so it's insertion ordered and should side-step the non-determinism issues you were seeing before.

(Not important for this PR IMO given that basis_gates often small, but perhaps a helpful pattern for the future.)

Well, the issue here is that we are constructing a target, so we cannot access its gate map yet... but I see your point, will keep it in mind :)

qiskit/providers/fake_provider/fake_generic.py

releasenotes/notes/add-generic-fake-backend-c1434e0c5c413935.yaml

test/python/compiler/test_transpiler.py

kevinhartman

LGTM, thanks for seeing this through @ElePT and @MozammilQ for your contributions.

ElePT

The approval was dismissed after fixing a last-minute merge conflict, so I'm re-approving to get it back in the queue :)

kevinhartman · 2024-01-29T20:02:50Z

Looks like another merge issue. I'll resolve this and push shortly.

jakelishman · 2024-01-29T20:28:23Z

The problems are likely a minor logical merge conflict with new tests added in #11554, and no inherent problem.

ElePT · 2024-01-29T22:27:24Z

@Mergifyio backport stable/0.46

mergify · 2024-01-29T22:27:51Z

backport stable/0.46

✅ Backports have been created

#11670 Add fake generic and modify tests (backport #10266) has been created for branch stable/0.46 but encountered conflicts

* Added a FakeGeneric BackendV2 backend, this is just a bare minimum working code * All FakeBackends (like FakeMelbourne, FakeBoeblingen ,etc) is replaced with FakeGeneric * relocated FakeGeneric in fake_provider directory; replaced if-elif with instruction_dict in fake_generic.py; added support of grid type of coupling map; using lower number of qubits, passed all tests of test_transpiler except one method ( test_parallel_dispatch_lazy_cal_loading ) * added tests for FakeGeneric * This commit just reformats test_transpiler.py fake_generic.py and test_fake_generic.py * Update qiskit/providers/fake_provider/fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Update qiskit/providers/fake_provider/fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Update test/python/providers/fake_provider/test_fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Update test/python/providers/fake_provider/test_fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Fix some tests * Add calibration schedules, fix transpiler tests * Fix lint * Refactor, add pulse functionality * Latest updates to FakeGeneric * Fix some unit tests * Fix some unit tests * Fix tests, lint, refactor * Remove unused inputs * Make calibrations optional * Attempt to speed up test: only add calibrations if option enabled * Revert some tests to V1 * Refactor FakeGeneric * Update unit test * Rename supported instructions to operations, fix tests Fix test seed Fix unit test, lint * Restore vf2postlayout test to V1 Fx vf2 test * Add pulse test to validate pulse capabilities of new fake backend Update fake generic * Update docs, fix lint * Avoid set for basis gates to allow reproducibility in the error/duration value generation * Update calibrations * Fix transpiler test * Add run test to confirm noise defaults * Add CZ to basis gates, dtm, update backend names, fix docs * Remove GenericTarget, add default for basis_gates, remove supported_operations. * Apply review comments, modify calibrate_instructions to avoid public calibration method. * New name proposal * Fix lint and test * Fix comment * Apply Kevin's suggestions to extract defaults from class * Make defaults private * Fix lint * Revert previous 3 comments. * Privatize GenericFakeBackend * Remove from docs * Revert privatization * Apply review comments * Add reno * Fix lint * Apply suggestions from Matt's code review Co-authored-by: Matthew Treinish <mtreinish@kortar.org> * Apply comments from code review * Fix lint * Rename to GenericBackendV2 and adjust docs. * Fix lint * Apply comments from code review * It's not 2023 anymore * Fix conflict * Fix test conflict. --------- Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com> Co-authored-by: Elena Peña Tapia <epenatap@gmail.com> Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Kevin Hartman <kevin@hart.mn> (cherry picked from commit b4582a9) # Conflicts: # qiskit/providers/fake_provider/__init__.py # test/python/compiler/test_transpiler.py

* Add fake generic and modify tests (#10266) * Added a FakeGeneric BackendV2 backend, this is just a bare minimum working code * All FakeBackends (like FakeMelbourne, FakeBoeblingen ,etc) is replaced with FakeGeneric * relocated FakeGeneric in fake_provider directory; replaced if-elif with instruction_dict in fake_generic.py; added support of grid type of coupling map; using lower number of qubits, passed all tests of test_transpiler except one method ( test_parallel_dispatch_lazy_cal_loading ) * added tests for FakeGeneric * This commit just reformats test_transpiler.py fake_generic.py and test_fake_generic.py * Update qiskit/providers/fake_provider/fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Update qiskit/providers/fake_provider/fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Update test/python/providers/fake_provider/test_fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Update test/python/providers/fake_provider/test_fake_generic.py Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> * Fix some tests * Add calibration schedules, fix transpiler tests * Fix lint * Refactor, add pulse functionality * Latest updates to FakeGeneric * Fix some unit tests * Fix some unit tests * Fix tests, lint, refactor * Remove unused inputs * Make calibrations optional * Attempt to speed up test: only add calibrations if option enabled * Revert some tests to V1 * Refactor FakeGeneric * Update unit test * Rename supported instructions to operations, fix tests Fix test seed Fix unit test, lint * Restore vf2postlayout test to V1 Fx vf2 test * Add pulse test to validate pulse capabilities of new fake backend Update fake generic * Update docs, fix lint * Avoid set for basis gates to allow reproducibility in the error/duration value generation * Update calibrations * Fix transpiler test * Add run test to confirm noise defaults * Add CZ to basis gates, dtm, update backend names, fix docs * Remove GenericTarget, add default for basis_gates, remove supported_operations. * Apply review comments, modify calibrate_instructions to avoid public calibration method. * New name proposal * Fix lint and test * Fix comment * Apply Kevin's suggestions to extract defaults from class * Make defaults private * Fix lint * Revert previous 3 comments. * Privatize GenericFakeBackend * Remove from docs * Revert privatization * Apply review comments * Add reno * Fix lint * Apply suggestions from Matt's code review Co-authored-by: Matthew Treinish <mtreinish@kortar.org> * Apply comments from code review * Fix lint * Rename to GenericBackendV2 and adjust docs. * Fix lint * Apply comments from code review * It's not 2023 anymore * Fix conflict * Fix test conflict. --------- Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com> Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com> Co-authored-by: Elena Peña Tapia <epenatap@gmail.com> Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Kevin Hartman <kevin@hart.mn> (cherry picked from commit b4582a9) # Conflicts: # qiskit/providers/fake_provider/__init__.py # test/python/compiler/test_transpiler.py * Update __init__.py * Update test_transpiler.py * Update test_transpiler.py * Fix conflicts (#11673) * Rename test, switch to basic simulator * Fix bug in pulse default generation --------- Co-authored-by: MozammilQ <54737489+MozammilQ@users.noreply.github.com> Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>

MozammilQ added 5 commits June 6, 2023 15:31

Added a FakeGeneric BackendV2 backend, this is just a bare minimum wo…

060d56b

…rking code

All FakeBackends (like FakeMelbourne, FakeBoeblingen ,etc) is replace…

ae26ee5

…d with FakeGeneric

added tests for FakeGeneric

7f544d4

This commit just reformats test_transpiler.py fake_generic.py and tes…

c864645

…t_fake_generic.py

MozammilQ requested review from a team and jyu00 as code owners June 12, 2023 17:41

qiskit-bot added the Community PR PRs from contributors that are not 'members' of the Qiskit repo label Jun 12, 2023

MozammilQ commented Jun 12, 2023

View reviewed changes

test/python/compiler/test_transpiler.py Outdated Show resolved Hide resolved

HuangJunye self-assigned this Jun 13, 2023

atharva-satpute suggested changes Jul 1, 2023

View reviewed changes

MozammilQ and others added 4 commits July 1, 2023 17:19

Update qiskit/providers/fake_provider/fake_generic.py

1531e84

Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com>

Update qiskit/providers/fake_provider/fake_generic.py

bf91f35

Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com>

Update test/python/providers/fake_provider/test_fake_generic.py

c34316a

Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com>

Update test/python/providers/fake_provider/test_fake_generic.py

dc24618

Co-authored-by: atharva-satpute <55058959+atharva-satpute@users.noreply.github.com>

ElePT and others added 3 commits October 2, 2023 16:12

Merge branch 'main' into add_FakeGeneric_and_modify_tests

0e57867

Fix some tests

02099a0

Add calibration schedules, fix transpiler tests

680c3ce

ElePT reviewed Oct 2, 2023

View reviewed changes

qiskit/providers/fake_provider/fake_generic.py Outdated Show resolved Hide resolved

Fix lint

6131e31

This was referenced Oct 3, 2023

Add generic Fake Backends and update use in Qiskit unit tests #10954

Closed

Update remaining unit tests to use FakeGeneric #10918

Merged

ElePT added 2 commits January 27, 2024 15:08

Fix lint

fe37393

Rename to GenericBackendV2 and adjust docs.

a36acaf

Fix lint

f1816ea

ElePT requested review from mtreinish and jakelishman January 29, 2024 15:51

kevinhartman reviewed Jan 29, 2024

View reviewed changes

ElePT added 2 commits January 29, 2024 18:49

Apply comments from code review

741831e

It's not 2023 anymore

ee588cc

ElePT requested a review from kevinhartman January 29, 2024 18:00

kevinhartman previously approved these changes Jan 29, 2024

View reviewed changes

kevinhartman enabled auto-merge January 29, 2024 18:07

ElePT and others added 2 commits January 29, 2024 19:22

Merge branch 'main' into add_FakeGeneric_and_modify_tests

93cdb36

Fix conflict

cba3eba

ElePT dismissed kevinhartman’s stale review via cba3eba January 29, 2024 19:26

ElePT previously approved these changes Jan 29, 2024

View reviewed changes

Fix test conflict.

72d1dee

kevinhartman dismissed ElePT’s stale review via 72d1dee January 29, 2024 20:29

ElePT approved these changes Jan 29, 2024

View reviewed changes

kevinhartman added this pull request to the merge queue Jan 29, 2024

Merged via the queue into Qiskit:main with commit b4582a9 Jan 29, 2024
12 checks passed

mergify bot mentioned this pull request Jan 29, 2024

Add fake generic and modify tests (backport #10266) #11670

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fake generic and modify tests #10266

Add fake generic and modify tests #10266

MozammilQ commented Jun 12, 2023 •

edited by ElePT

qiskit-bot commented Jun 12, 2023

MozammilQ commented Jun 12, 2023

HuangJunye commented Jun 13, 2023

atharva-satpute left a comment

MozammilQ commented Jul 1, 2023

HuangJunye commented Oct 2, 2023

MozammilQ commented Oct 2, 2023

MozammilQ commented Oct 2, 2023

HuangJunye commented Oct 2, 2023

ElePT left a comment

ElePT commented Jan 27, 2024 •

edited

ElePT commented Jan 29, 2024

kevinhartman left a comment

kevinhartman Jan 29, 2024

kevinhartman Jan 29, 2024

ElePT Jan 29, 2024

kevinhartman left a comment

ElePT left a comment •

edited

kevinhartman commented Jan 29, 2024

jakelishman commented Jan 29, 2024

ElePT commented Jan 29, 2024

mergify bot commented Jan 29, 2024 •

edited

Add fake generic and modify tests #10266

Add fake generic and modify tests #10266

Conversation

MozammilQ commented Jun 12, 2023 • edited by ElePT

Summary

Details and comments

qiskit-bot commented Jun 12, 2023

MozammilQ commented Jun 12, 2023

HuangJunye commented Jun 13, 2023

atharva-satpute left a comment

Choose a reason for hiding this comment

MozammilQ commented Jul 1, 2023

HuangJunye commented Oct 2, 2023

MozammilQ commented Oct 2, 2023

MozammilQ commented Oct 2, 2023

HuangJunye commented Oct 2, 2023

ElePT left a comment

Choose a reason for hiding this comment

ElePT commented Jan 27, 2024 • edited

ElePT commented Jan 29, 2024

kevinhartman left a comment

Choose a reason for hiding this comment

kevinhartman Jan 29, 2024

Choose a reason for hiding this comment

kevinhartman Jan 29, 2024

Choose a reason for hiding this comment

ElePT Jan 29, 2024

Choose a reason for hiding this comment

kevinhartman left a comment

Choose a reason for hiding this comment

ElePT left a comment • edited

Choose a reason for hiding this comment

kevinhartman commented Jan 29, 2024

jakelishman commented Jan 29, 2024

ElePT commented Jan 29, 2024

mergify bot commented Jan 29, 2024 • edited

✅ Backports have been created

MozammilQ commented Jun 12, 2023 •

edited by ElePT

ElePT commented Jan 27, 2024 •

edited

ElePT left a comment •

edited

mergify bot commented Jan 29, 2024 •

edited