This repository was archived by the owner on Sep 20, 2022. It is now read-only.
Repository navigation
This repository was archived by the owner on Sep 20, 2022. It is now read-only.
Allow to use an existing test report instead of creating a new one #63
Description
I’m running gemini test as part of my CI pipeline in order to prevent deploying unwanted visual changes. This works quite well. I also want to use the gemini-gui to make it easy to accept intentional changes. The problem here is that I have to run the tests again in the GUI instead of reusing the existing report made by the CI pipeline.
Ideally there would be even an option to disable running tests via the GUI as this could produce a different result than the test run before that was executed by the CI pipeline (e.g. if new commits have been pushed).
Reacted by Ruslan Ibrahimau and generalov
Reacted by Ruslan Ibrahimau and generalov
Activity
sipayRT commentedon Sep 16, 2016
sorry, I don't understand how can you reuse the existing report. Is that report was generated by
gemini testcommand?lo1tuma commentedon Sep 16, 2016
@sipayRT Exactly. I’m using
gemini testthe create a a test report which I then want to use ingemini-gui.SevInf commentedon Sep 16, 2016
Actually, our team will also be interested in this feature.
The way it can be possibly implemented:
{ "mySuteName": { "children": { "subSuiteName": ... }, "browsers": { "chrome": { "image": "{base64-encoded image}" }, "ie": { "error": "stacktrace" } } } }Then CI system can publish new
jsonreport alongsidehtmland developers will be able to update the images without re-running the tests.j0tunn commentedon Sep 26, 2016
@SevInf I think this could be done via some
gemini-json-reporterplugin.Also this plugin can emulate gemini events on load when started with some command line option (something like
--play).gemini-guiwill receive those events and display all the stuff.What do you think?
SevInf commentedon Sep 27, 2016
@j0tunn yes, plugin might work.
Does it mean we need to introduce a programmatic way to launch the
guiwith injectedgeminiinstance?lo1tuma commentedon Sep 27, 2016
I have implemented a custom reporter which generates json but I've decided to go with a flat array instead of the tree structure suggested above.
I also added a hash value to each test result so that it can be uniquely identified. This allows you to make changes to the result e. g. via a GUI.
As a proof of concept I am currently trying to implement a separate GUI which less features than this one. It only allows you to see the test results and approving visual changes. As input / config it takes the json file from my custom reporter and the path to the reference images.
I hope to have something ready to publish soon.
j0tunn commentedon Sep 27, 2016
No, gemini-gui and plugin will share the same gemini instance
lo1tuma commentedon Oct 12, 2016
I’ve published the json reporter and alternative GUI.
Feedback would be very welcomed.
SevInf commentedon Oct 12, 2016
@lo1tuma wow, awesome work!
I was not able to check it our environment since our CI is still on node 4, but I like the polished looks of the UI and I'm pretty convinced that this is how main GUI and HTML report should look like.
Will you be interested in merging it into mainline?
lo1tuma commentedon Oct 13, 2016
@SevInf You could run the GUI in a docker container and simply mount the folder where the reference screenshots are located, then the node 6 dependency shouldn’t be a problem. But I think the json-reporter needs node 6 as well.
I would be definitely interested in merging this into the mainline GUI, but currently the feature set is quite different and I’m not sure if there is a good way to support both feature sets in the same GUI. Let me know if you have some good ideas about this.