First commit

Author: Felix Stamm

.gitattributes

@@ -0,0 +1 @@
+*.ipynb filter=jupyternotebook

.gitconfig

@@ -0,0 +1,3 @@
+[filter "jupyternotebook"]
+	clean = python clean_errors.py %f
+	required

.github/workflows/pythonapp.yml

@@ -0,0 +1,42 @@
+# This workflow will install Python dependencies, run tests and lint with a single version of Python
+# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
+
+name: build
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.8", "3.9", "3.10"]
+
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install .
+
+      - name: Test with pytest
+        run: |
+          pip install pytest
+          pytest
+
+#    - name: Lint with flake8
+#      run: |
+#        pip install flake8
+#        # stop the build if there are Python syntax errors or undefined names
+#        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+#        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
+#        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

.gitignore

@@ -0,0 +1,154 @@
+/datasets/*
+/scripts/images
+/scripts/results/
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+scripts/Katz convergence.py
+scripts/ca-netscience.edges
+scripts/ERGM.pkl
+tmp.txt
+scripts/convergence2_eigenvector.py
+scripts/convergence2_hits.py
+scripts/convergence2_jaccard.py
+scripts/convergence2_katz.py
+scripts/convergence2_pagerank.py
+scripts/datasets_path.txt
+env
+nestmodel.egginfo

.pylintrc

@@ -0,0 +1,8 @@
+[MASTER]
+disable= missing-module-docstring
+[FORMAT]
+max-line-length=200
+
+# allow 1 or 2 length variable names
+good-names-rgxs=^[_a-zGW][_a-z0-9L]?$
+good-names=WL_fast,WL_both, M, A, A0, A1, A_work, M_in, C1, C2, WL1, WL2, SAE, ERGM, ERGM_experiments, G_str

README.md

@@ -0,0 +1,73 @@
+# Neighborhood Structure Configuration Model
+![Teaser Figure for the Neighborhood Structure Configuration Model](./teaser.png)
+
+![build](https://github.com/Feelx234/nestmodel-test/actions/workflows/pythonapp.yml/badge.svg)
+
+Neighborhood Structure Configuration Models (short NeSt-models) allow to generate random graphs that preserved the neighborhood structure of a given graph for arbitrary depth d.
+
+This package provides the implementation as used in our paper.
+
+
+# Installation instructions
+You can install this library like any other python package, depending packages are also installed along with it.
+
+
+### Installing nestmodel
+```
+git clone https://github.com/Feelx234/nestmodel.git
+pip install nestmodel
+```
+The installation should take less than a minute.
+If you also want to notebooks in the scripts folder please install `jupyter` as outlined below.
+```
+pip install jupyter
+```
+
+### Making sure the installation went smoothly
+
+To make sure the installation has succeeded we need to install pytest as well and then run the tests
+```
+pip install pytest
+python -m pytest nestmodel
+```
+If you do not have the graph tools library installed, the tests will emit a warning that it is not installed which can be ignored for now.
+
+# Reproducing the results from the paper
+
+Before reproducing the results you need to get the **datasets**. We provide a download script `/scripts/get_datasets.py` which will download all datasets for you.
+```
+cd nestmodel
+python ./scripts/get_datasets.py
+```
+
+To reproduce the results from the paper we provided different scripts. The detailed convergence results on HepPh are available in `/scripts/convergence2_example.ipynb`.
+This file can be opened in jupyter and should work out of the box.
+
+
+The convergence results in sum absolute error (SAE) can all be produced by using
+```python ./scripts/convergence2_runner.py -all```
+The runtime of this script is about 1 day.
+The script will first convert the notebook files into `.py` files and then run tests on the karate network which will only take about 1 min.
+Once the script is run, plots can be created by opening the `convergence2_plot.ipynb` file in jupyter. The plot file requires the time of completion for the different centrality calculations which can be read of from the result files which appear in `./scripts/results/` and are also stored in a file: `./scripts/convergence2_run_results.txt`.
+
+If you are interested in convergence results for individual centralities you can also checkout any of the `./scripts/convergence2_XXXXX.ipynb` (for example `./scripts/convergence2_pagerank.ipynb`) in jupyter.
+
+
+
+The runtime results comparing NeSt with ERGM and Erdös Renyi models can be reproduced using
+```python ./scripts/Baselines.ipynb```
+Notice that the runtime of the Baselines file for the phonecalls is *several days* because the ERGM is so slow! The baseline results can be visualized in the `./scripts/Baselines_plot.ipynb` notebook.
+
+
+
+### System requirements
+The package should run on any modern standard computer.
+
+
+### Software requirements
+#### OS Requirements
+This package has been tested on *Linux*.
+
+#### Python Dependencies
+The package was tested with various version of python (>=3.8). It depends on other packages that are automatically installed along with nestmodel and can be looked up at the setup.py file.
+Parts of it also depend on the just-in-time compiler `numba` we used version 0.56.4 but should work on other versions as well.

clean_errors.py

@@ -0,0 +1,47 @@
+import json
+import sys
+from pathlib import Path
+
+# <<< encoding UTF8 !!!!! >>>
+#print(sys.argv)
+
+j=None
+if len(sys.argv[1])<2:
+    print("No file provided")
+    exit()
+
+path = Path(sys.argv[1])
+with open(path, "r", encoding="utf-8") as f:
+    j = json.load(f)
+
+destination = path
+
+#print(len())
+cells = j["cells"]
+deleted = 0
+cells = j["cells"]
+deleted = 0
+for i in range(len(cells)-1, -1, -1):
+    cell = cells[i]
+    if cell["cell_type"] == "code":
+        if len(cell["outputs"])>0:
+            to_remove = []
+            for i, output in enumerate(cell["outputs"]):
+                if "name" in output and output["name"]=="stderr":
+                    to_remove.append(i)
+            to_remove = reversed(to_remove)
+            for i in to_remove:
+                cell["outputs"].pop(i)
+                deleted+=1
+#print(f"deleted {deleted} cells")
+destination = path# path.with_name(path.stem + "_stripped.ipynb")
+#print(f"deleted {deleted} cells")
+#destination = path.with_name(path.stem + "_stripped.ipynb")
+#destination = path.with_stem(path.stem + "_stripped")
+#print(f"writing to {destination}")
+if deleted > 0:
+    with open(destination, 'w', encoding="utf-8") as f:
+        json.dump(j, f, indent=1)
+        f.write("\n")
+
+print(json.dumps(j, indent=1))