Return to Answer

After a short exploratory meeting with Alation, was pointed towards Data Cookbook which looks promising (and apparently is used by many other universities). Will look into further and may update this answer.

http://www.datacookbook.com/videos-2/ (as a DE, I'd recommend watching the videos in order: 1, 2, 5, 4, 3)

1. After watching the pre-recorded demo videos, I think it actually looks pretty good and could be what we need for our data cataloging needs (unlike the other tech/datasci. focused catalog systems I've seen with Erwin, Alation, Collibra, Amundsen, et al which focus more on structured data for more tech-centric users). Appears to be used by many other universities (and MSU, PSU, Birmingham U, etc.) (confirming what I had been told when initially pointed to them by the Alation people I met with). More details can be found here in this evaluation by the University of Birmingham on Data Cookbook for data governance, cataloging, and reporting: https://intranet.birmingham.ac.uk/it/innovation/documents/public/Experiments/TheDataCookBook-Evaluation-v0.1.pdf

2. The marketed pricing (http://www.datacookbook.com/pricing/) appears not as expensive as some other price tags I've seen

3. I'd also note that Lyft's Amundsen could be a good open source internal data catalog system for our own data analysts for cataloging and making-searchable our internal structured/tabled data assets.

4. A major question that I was still not able to discern was whether certain Defs/Specs could be hidden from certain users / only visible to a subset of catalog users. It appears so

After a short exploratory meeting with Alation, was pointed towards Data Cookbook which looks promising (and apparently is used by many other universities). Will look into further and may update this answer.

https://www.datacookbook.com/videos-2/ (as a DE, I'd recommend watching the videos in order: 1, 2, 5, 4, 3)

1. After watching the pre-recorded demo videos, I think it actually looks pretty good and could be what we need for our data cataloging needs (unlike the other tech/datasci. focused catalog systems I've seen with Erwin, Alation, Collibra, Amundsen, et al which focus more on structured data for more tech-centric users). Appears to be used by many other universities (and MSU, PSU, Birmingham U, etc.) (confirming what I had been told when initially pointed to them by the Alation people I met with). More details can be found here in this evaluation by the University of Birmingham on Data Cookbook for data governance, cataloging, and reporting: https://intranet.birmingham.ac.uk/it/innovation/documents/public/Experiments/TheDataCookBook-Evaluation-v0.1.pdf

2. The marketed pricing (https://www.datacookbook.com/pricing/) appears not as expensive as some other price tags I've seen

3. I'd also note that Lyft's Amundsen could be a good open source internal data catalog system for our own data analysts for cataloging and making-searchable our internal structured/tabled data assets.

4. A major question that I was still not able to discern was whether certain Defs/Specs could be hidden from certain users / only visible to a subset of catalog users. It appears so

To a much lesser extent, CKAN or Magda could also work. Magda looks like a good candidate for this, though I question the current state of the user access restriction capabilities and CKAN currently lacks any moderation features like iData has. Note that magda requires either using Google Cloud services to host (uses Terraform) or having your own kubernetes cluster to host it on. Also, I see that the AU gov uses it, but when you try to sign in or create an account, it takes you to a site that uses CKAN. I wonder what the difference is between CKAN and Magda?