Copyright (c) 2018 Universidade Federal Fluminense (UFF).
The PyPOFacets project provides Python implementations of the Radar Cross Section (RCS) simulation. Our implementations are Python translations of the original MATLAB POFacets implementation.
This is the companion website of the paper On the Importance of Modularization to the Meaningfulness of Provenance from Scripts (Journal of Information and Data Management). It contains the scripts, provenance graphs, prompts, and analysis materials used in the paper. All noWorkflow graphs were captured with noWorkflow 2.1.1 on CPython 3.13 and exported with now export dataflow <trial> -d 1.
- Script
- Provenance graph generated by noWorkflow (1,359 nodes, 5,374 edges; Figure 4 of the paper)
- Script
- Provenance graph generated by YesWorkflow (ground truth; Figure 3 of the paper)
- Script (14 functions, one per YesWorkflow block, obtained with Extract Method)
- Provenance graph generated by noWorkflow (69 nodes, 110 edges); Figure 6 of the paper shows a derived view with only the function-call and file nodes (output/provenance/reduced)
- Script (8 functions, generated by Gemini from the monolithic script)
- Provenance graph generated by noWorkflow (55 nodes, 82 edges), regenerated from the published script, which writes its output files under
output/modular_llm/; Figure 7 of the paper shows a derived view with only the function-call and file nodes (output/provenance/reduced)
- Prompt used to obtain the LLM-modularized PyPOFacets script. The same prompt was given to ChatGPT, Gemini, and GitHub Copilot in February 2026; the Gemini version was selected for the paper (Section 4.5).
- Prompts of the multi-script evaluation: demodularization (expert script to monolithic baseline) and remodularization (monolithic baseline to LLM-modularized version), both run with Claude Opus 4.6 through GitHub Copilot Chat in Visual Studio Code (Section 7).
- llm_supplementary_round/: on 28 October 2026, the prompt above was sent again, once to ChatGPT, Gemini, and GitHub Copilot Chat, and all three outputs were evaluated without selection. The folder records the configuration each interface presented at the time and contains the outputs as generated, the evaluated copies with their two edits (the same as for the Gemini version), and their provenance graphs; the results are in analysis/results (files
llm_round_*).
- mappings.xlsx: correspondence between the YesWorkflow ground truth (40 data items, 7 port nodes, 14 blocks, data-flow edges) and the names used in each version. Row 10 of sheet Edges was corrected from
ReadArgs -> TimetoReadArgs -> OutputDir, as stated by the YesWorkflow annotation ofoutput_dir(line 33 ofyw_monolithic_pypofacets.py). The sheet Analysis gives the values reported in the revised paper (Table 3 with both components, Figure 8, the edge comparison, and the robustness check), read from analysis/results; the sheet Analysis_2018 keeps the original computation over the graphs of the first submission and is superseded. - analysis/: the scripts that compute the comparisons of the paper (
table3_recompute.py,edges_compare.py,label_match.py,compare_functions.py,compare_functions_lib.py,sim_per_script.py,reduced_view.py) and their outputs inanalysis/results/:- Table 3:
table3.txtand one trail per version (table3_<version>.csv: each ground-truth item, its status and the node that matches it). A node matches an item only when its label equals the item's name (file nodes match by path), one node per item. The manifest of manual matches by final value (aliases.csv) is empty: no admissible match was found in the noWorkflow databases of the three versions. - Figure 8: the 16 nodes of the monolithic graph determined by this rule (
fig8_monolithic_exact_matches.csv) and the counts over all nodes of the graph (figure8_monolithic_universe_all.txt). - Edge comparison:
edges_expert.txt/.csv(95 ground-truth edges, 89 matched, one-to-one, orientation ignored) andedges_monolithic.txt/.csv(none matched). - Functional alignment per script of the multi-script evaluation (Section 7):
sim_per_script.txt/.csv. - Robustness check of Section 9:
robustness_table3_variables.txt. - Supplementary LLM round of Section 9 (Table 10): files
llm_round_*(see llm_supplementary_round/). Node identifiers (e_N,a_N) are specific to each execution; the trails refer to the graphs inoutput/provenance.
- Table 3:
- compare_functions.py and similarity.txt: the functional-structure equivalence of the first submission (Jaccard similarity over tokenized lines, bidirectional matching); the functional component of Table 3 is now computed by
table3_recompute.pywith the same metric.
- Scripts of the six algorithms (expert, monolithic, and LLM-modularized versions) and the four scripts left for future work: experiments-jidm repository.
- multiscript/: re-execution package with the 18 provenance graphs (
dataflows/*.dot), the three PyPOFacets graphs, the execution records (runs.csv,environment.txt), the metrics behind Tables 4, 7, and 8 (metrics.csv), and the analysis scripts (run_all.py,graph_metrics.py,effect_sizes.py,capture_results.py,compare_results.py,table3_variables.py).
Reproducibility instructions are available on the PyPOFacets project wiki page.
Parts of the following software were used by PyPOFacets directly or in an adapted form:
POFacets
Copyright (c) 2012 David Jenn. US government work and not subject to copyright.
All rights reserved.
The MIT License (MIT)
Copyright (c) 2018 Universidade Federal Fluminense (UFF).
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.