sw_arch_doc: replace cinclude2dot with new python alternative. benefits:

- finer-grained control over the set of input directories
- output is more deterministic (deduplicated and sorted)

Author: pestophagous

build_app.sh

@@ -44,7 +44,7 @@ else
   MYAPP_EXTRA_CONF=""
   # adding this next line here (rather than in run_all_tests.sh), because we think
   # of the graph-generation as more of a "build" step than a "test":
-  $DIR/sw_arch_doc/generate_graph.sh ${DIR}/src/ ${DIR}/
+  $DIR/sw_arch_doc/generate_graph.sh -s ${DIR}/src/ -f ${DIR}/ -x $DIR/sw_arch_doc/excludes.txt
 
   # Next step done here (rather than inside generate_graph.sh) because the
   # graph script does not need to be "git aware".

build_cmake_app.sh

@@ -58,7 +58,7 @@ else
   MYAPP_EXTRA_CONF=()
   # adding this next line here (rather than in run_all_tests.sh), because we think
   # of the graph-generation as more of a "build" step than a "test":
-  $DIR/sw_arch_doc/generate_graph.sh ${DIR}/src/ ${DIR}/
+  $DIR/sw_arch_doc/generate_graph.sh -s ${DIR}/src/ -f ${DIR}/ -x $DIR/sw_arch_doc/excludes.txt
 
   # Next step done here (rather than inside generate_graph.sh) because the
   # graph script does not need to be "git aware".

cookiecutter/cookiecutter.json

@@ -7,6 +7,7 @@
   "email": "[email protected]",
   "year": "2021",
   "_copy_without_render": [
-    "*third_party"
+    "*third_party",
+    "sw_arch_doc"
   ]
 }

sw_arch_doc/README.md

@@ -2,15 +2,24 @@
 
 ### What is this?
 
-This is a wrapper script that simplifies usage of `cinclude2dot`.
+This folder has scripts for C/C++ dependency graphing.
 
-Credit for `cinclude2dot` belongs to https://www.flourish.org/cinclude2dot/
+These scripts use the [Graphviz](https://graphviz.org/) `dot` tool to produce an
+SVG image visualizing the header-include graph of a C/C++ codebase.
+
+You can apply these scripts to your entire C/C++ codebase, or you can visualize
+any subset of your code, bounding the input to specific subdirectories of your
+codebase.
+
+Original inspiration taken from https://www.flourish.org/cinclude2dot/, but here
+we use a novel implementation in python, with different cli options and slightly
+different output (though the output of both is in the `dot` graphviz language).
 
 You can copy this `sw_arch_doc` folder into any project (or just copy it
-somewhere on your system) and run the script against any folder of C/C++ source
-files you like.
+somewhere on your system) and run the script against any folder or set of
+folders of C/C++ source files you like.
 
-When choosing which folder of source files to diagram, we recommend not
+When choosing which folder(s) of source files to diagram, we recommend not
 attempting to diagram more than ~100 files at a time, or the diagram will be
 very dense and hard to read.
 
@@ -24,13 +33,18 @@ apply added styling to your graph(s).
 **TL;DR**
 
 ```
-generate_graph.sh ${PATH_TO_SOURCES} ${PATH_TO_HEADERS}
+./generate_graph.sh -s ${SOURCE_PATH_1} ${SOURCE_PATH_2} -x ${TEXT_FILE_WITH_YOUR_EXCLUDES}
 ```
 
 The generated diagram will be: `all_src.svg`
 
 **Required:** `dot` (from the graphviz package) must be installed on your sytem.
 
+For detailed help, run:
+```
+./generate_graph.sh --help
+```
+
 If you prefer to alter the appearance of the graph by using different (or
 additional) graph attributes, edit the file `dot_head.part` with the
 [graphviz attributes](https://graphviz.org/doc/info/attrs.html) that you
@@ -45,22 +59,22 @@ your preferred graph attributes.
 #### Exclusions
 
 Many projects have header files such as `defs.h` or `utils.h`, and it is usually
-desireable to exclude these headers from the generated diagram.
+desirable to exclude these headers from the generated diagram.
 
-That is the goal of the lines near the bottom of `generate_graph.sh` that look as follows:
+For any/all header(s) you wish to ignore and exclude from the visualization,
+list the *stem* of the header name in a file such as `excludes.txt`, listing one
+*stem* name per line. Pass your `excludes.txt` to the script like so:
 
 ```
-"${THESED}" -i "/\"\bcli_options\b\"/d" ${ARCH_DOC_DIR}/all_src.dot
-"${THESED}" -i "/\"\blogging_tags\b\"/d" ${ARCH_DOC_DIR}/all_src.dot
-"${THESED}" -i "/\"\bresource_helper\b\"/d" ${ARCH_DOC_DIR}/all_src.dot
+./generate_graph.sh -s ${SOURCE_PATH_1} ${SOURCE_PATH_2} -x excludes.txt
 ```
 
-Add more lines as needed for your project.
+See the example `excludes.txt` in this folder.
 
 For example, to remove `defs.h` from the diagram, you would add the following line:
 
 ```
-"${THESED}" -i "/\"\bdefs\b\"/d" ${ARCH_DOC_DIR}/all_src.dot
+defs
 ```
 
 ### My diagram looks like a tangle of spaghetti. Now what?
@@ -69,9 +83,9 @@ There are several reasons that can cause you to end up with spaghetti.
 
 First, we recommend not attempting to diagram more than ~100 files at a
 time. This means you should make sure that the number of C/C++ source files
-residing under whatever directory you chose as the `${PATH_TO_SOURCES}` argument
-does not exceed 100 total source files. When counting files under
-`${PATH_TO_SOURCES}`, any files in nested subdirectories count, too!
+residing under whatever directory or directories you chose as the `-s` (or
+long-name `--src-dir`) argument does not exceed 100 total source files. When
+counting files under `--src-dir`, any files in nested subdirectories count, too!
 
 Second, please recall the advice above regarding explicit exclusions for common
 files such as `defs.h`. If your generated diagram is large, you can usually spot
@@ -91,6 +105,6 @@ such as [deheader](http://www.catb.org/~esr/deheader/). After you identify and
 remove spurious `#include` lines from your sources, then re-run
 `generate_graph.sh` and you should see a smaller diagram.
 
-You might find that generating the diagram periodically (during CI, even!) gives
-you a good way to notice spurious `#include` edges and stay on top of your
+You might find that generating the diagram regularly (during every build, even!)
+gives you a good way to notice spurious `#include` edges and stay on top of your
 header hygiene.

sw_arch_doc/all_src.dot

@@ -4,16 +4,94 @@ digraph "source tree" {
     fontsize="16";
     fontname="Helvetica";
 	clusterrank="local";
-	"event_filter" -> "every_so_often"
-	"lib" -> "example_shared"
-	"lib" -> "quarantined_awaiting_warning_fixes"
-	"main" -> "ios_log_redirect"
-	"main" -> "logger"
-	"main" -> "view_model_collection"
-	"quarantined_awaiting_warning_fixes" -> "sample_header_exempt_from_warnings"
-	"timer_service_test" -> "timer_service"
-	"view_model_collection" -> "event_filter"
-	"view_model_collection" -> "gui_tests"
-	"view_model_collection" -> "lib"
-	"view_model_collection" -> "qml_message_interceptor"
+subgraph "cluster_box_dir_lib_app" {
+        label="lib_app";
+        "cli_options";
+}
+"cli_options"
+subgraph "cluster_box_dir_app" {
+        label="app";
+        "event_filter";
+}
+"event_filter"
+"event_filter" -> "every_so_often"
+subgraph "cluster_box_dir_util" {
+        label="util";
+        "every_so_often";
+}
+"every_so_often"
+subgraph "cluster_box_dir_lib_example_shared" {
+        label="lib_example_shared";
+        "example_shared";
+}
+"example_shared"
+subgraph "cluster_box_dir_app" {
+        label="app";
+        "gui_tests";
+}
+"gui_tests"
+subgraph "cluster_box_dir_lib_app" {
+        label="lib_app";
+        "lib";
+}
+"lib"
+"lib" -> "example_shared"
+"lib" -> "quarantined_awaiting_warning_fixes"
+subgraph "cluster_box_dir_lib_app" {
+        label="lib_app";
+        "logging_tags";
+}
+"logging_tags"
+"logging_tags" -> "cli_options"
+subgraph "cluster_box_dir_app" {
+        label="app";
+        "main";
+}
+"main"
+"main" -> "cli_options"
+"main" -> "view_model_collection"
+subgraph "cluster_box_dir_util" {
+        label="util";
+        "qml_message_interceptor";
+}
+"qml_message_interceptor"
+subgraph "cluster_box_dir_lib_app" {
+        label="lib_app";
+        "quarantined_awaiting_warning_fixes";
+}
+"quarantined_awaiting_warning_fixes"
+subgraph "cluster_box_dir_libstyles" {
+        label="libstyles";
+        "resource_helper";
+}
+"resource_helper"
+subgraph "cluster_box_dir_util" {
+        label="util";
+        "timer_service";
+}
+"timer_service"
+subgraph "cluster_box_dir_tests" {
+        label="tests";
+        "timer_service_test";
+}
+"timer_service_test"
+"timer_service_test" -> "timer_service"
+subgraph "cluster_box_dir_util" {
+        label="util";
+        "usage_log_t";
+}
+"usage_log_t"
+subgraph "cluster_box_dir_app" {
+        label="app";
+        "view_model_collection";
+}
+"view_model_collection"
+"view_model_collection" -> "cli_options"
+"view_model_collection" -> "event_filter"
+"view_model_collection" -> "gui_tests"
+"view_model_collection" -> "lib"
+"view_model_collection" -> "logging_tags"
+"view_model_collection" -> "qml_message_interceptor"
+"view_model_collection" -> "resource_helper"
+"view_model_collection" -> "usage_log_t"
 }