Filter Out Commits That Don't Belong to Any Branch

[ziadhany]

This commit does not belong to any branch in this repository and may belong to a fork outside the repository:

• ansible/ansible@fe28767

However, it is still listed as the fixing commit in both OSV and NVD:

• https://osv.dev/vulnerability/CVE-2021-3620
• https://nvd.nist.gov/vuln/detail/CVE-2021-3620
• https://public.vulnerablecode.io/advisories/github_osv/GHSA-4r65-35qq-ch8j

$ git branch -r --contains fe28767970c8ec62aabe493c46b53a5de1e5fac0
error: no such commit fe28767970c8ec62aabe493c46b53a5de1e5fac0

[ziadhany]

I ran a quick script to filter out invalid commits.

See more examples of some of the invalid commits here:

https://github.com/ansible/ansible/commit/fe28767970c8ec62aabe493c46b53a5de1e5fac0
https://github.com/assimp/assimp/commit/ab66a1674fcfac87aaba4c8b900b315ebc3e7dbd
https://github.com/cisco-talos/clamav/commit/fd0c4bc251c211b211b3e4f1813bd0d07bdba657
https://github.com/cog-creators/red-discordbot/commit/e269ea0d3bc88417163c18431b1df38a9be92bfc
https://github.com/cog-creators/red-discordbot/commit/9ab536235bafc2b42c3c17d7ce26f1cc64482a81
https://github.com/datadog/guarddog/commit/a56aff58264cb6b7855d71b00dc10c39a5dbd306
https://github.com/fedora-python/lxml_html_clean/commit/c5d816f86eb3707d72a8ecf5f3823e0daa1b3808
https://github.com/flask-middleware/flask-security/commit/61d313150b5f620d0b800896c4f2199005e84b1f
https://github.com/flask-middleware/flask-security/commit/6d50ee9169acf813257c37b75babe9c28e83542a
https://github.com/fluent/fluent-bit/commit/7e6e42d1c12644b6d5007b0415553e24761e7e59
https://github.com/fluent/fluent-bit/commit/5375fb219ca8aa70ecce8136621016a51c0b37ba
https://github.com/fluent/fluent-bit/commit/715e40c3104db7576866041e47bf62ccfbf8b518
https://github.com/fluent/fluent-bit/commit/c5741b48c24aa80ddd820e6f42e6ce8fc5f5d98b
https://github.com/fluent/fluent-bit/commit/2beca980f55714f669b6605f6b110e25522c1dbc
https://github.com/geonode/geonode/commit/2fdfe919f299b21f1609bf898f9dcfde58770ac0
https://github.com/geopython/owslib/commit/b92687702be9576c0681bb11cad21eb631b9122f
https://github.com/google/fuzzing/commit/5997a4fc09163c1baa186f5a9d00c4c8668a72b1
https://github.com/gpac/gpac/commit/be6e8a95e6fe02267f4960adffb3e6115bd5cb4a
https://github.com/gpac/gpac/commit/714810a4e158adf0c4a207637720ac08600adf41
https://github.com/gpac/gpac/commit/34b8d0aa8c14d59cbbfe9bd140f413fa19bf4b62
https://github.com/grokimagecompression/grok/commit/a7fefd0c8be45aa54ec721e1a7f29c2b5d13da04
https://github.com/grokimagecompression/grok/commit/a27b85d5d31dc6da58aa8af859e63448a6384150
https://github.com/grokimagecompression/grok/commit/0aa92a52735837cfb35ed7890747ce6dcfabf38e
https://github.com/grokimagecompression/grok/commit/746b605d5502e304a5aaafe61bf7807ee303f3ca
https://github.com/grokimagecompression/grok/commit/6b45934c6d0682de97d8e9652da85213648f9df8
https://github.com/grokimagecompression/grok/commit/7113ab221774f06eee06a361cfb0870b572fe0dd
https://github.com/grokimagecompression/grok/commit/f33b49fa40b0abad23f25d2c8a74794bccd22a53
https://github.com/grokimagecompression/grok/commit/73ef6711005759b4dc91381f6cdf1cba6d36b59f
https://github.com/grokimagecompression/grok/commit/faa589e382bf7bd63f24e2042210db3ee912227f
https://github.com/grokimagecompression/grok/commit/48d2c98b803c5e2ddddaa6281ba02aa7adc4777d
https://github.com/grokimagecompression/grok/commit/edecdb89539e32c31b701ba2cf6140737135697a
https://github.com/grokimagecompression/grok/commit/2750da0c384c29eedd23d99ee205949447fadba8
https://github.com/grokimagecompression/grok/commit/0cf697ed3f32dbf2df822a8a42974e50262b064d
https://github.com/grokimagecompression/grok/commit/39670926d0334190a31e3a2526137d97440973e3
https://github.com/grokimagecompression/grok/commit/6a364d7d81e07a9fdbdce74f27803288158d7a49
https://github.com/grokimagecompression/grok/commit/12ef5ef65d81862f0dcad9fafdbabb3c93a72934
https://github.com/hdfgroup/hdf5/commit/ff3d6722a91587daaaac82e78b25d26ad3f50172
https://github.com/hdfgroup/hdf5/commit/54f404b5ad8e63d99e3283646b543b2842a22fd3
https://github.com/invoke-ai/invokeai/commit/756008dc5899081c5aa51e5bd8f24c1b3975a59e
https://github.com/irmen/pyro3/commit/554e095a62c4412c91f981e72fd34a936ac2bf1e
https://github.com/jupyterhub/n/commit/07690644f29a566011dd0d7ba14cae3eb0490481
https://github.com/libgit2/li/commit/6de3221debf785c328842667eb04c07c747c73a0
https://github.com/libgit2/li/commit/b899fda3d88dc92f50e73544fb7524a1c3c70354
https://github.com/libgit2/li/commit/b8fff35ab32a384f7e224d1afd06ef4fe1863e1f
https://github.com/libgit2/li/commit/11c89c386c8862f834097cdc5d9ed79634cf00f3
https://github.com/libical/libical/commit/f8caa028c9c1beed11297c9dc8b1c7cfb76b454c
https://github.com/libical/libical/commit/1442c8a22259e7963573337a7c9d30baf64beee7
https://github.com/libredwg/libredwg/commit/0156c780ac6b109d191975041def0a258b0b75fc
https://github.com/martinpitt/python-dbusmock/commit/4e7d0df9093
https://github.com/mlflow/mlflow/commit/b43e0e3de5b500554e13dc032ba2083b2d6c94b8
https://github.com/mlflow/mlflow/commit/7162a50c654792c21f3e4a160eb1a0e6a34f6e6e
https://github.com/mlflow/mlflow/commit/96f0b573a73d8eedd6735a2ce26e08859527be07
https://github.com/mlflow/mlflow/commit/149c9e18aa219bc47e86b432e130e467a36f4a17
https://github.com/net-snmp/net-snmp/commit/5ab7cd628bf79f37a413715f5688ec82bf5cca13
https://github.com/net-snmp/net-snmp/commit/644bc1d4d155f464db94ba3f21be7eb2448f95f7
https://github.com/net-snmp/net-snmp/commit/90aa2c37dc6a8bc280951ec6b6efe9714b0ef4de
https://github.com/nuitka/nuitka/commit/09647745d7cbb6ff32f9fa948f19d5558b32bcad
https://github.com/openzeppelin/cairo-contracts/commit/6d4cb750478fca2fd916f73297632f899aca9299
https://github.com/parisneo/lollms/commit/95ad36eeffc6a6be3e3f35ed35a384d768f0ecf6
https://github.com/parisneo/lollms/commit/28ee567a9a120967215ff19b96ab7515ce469620
https://github.com/parisneo/lollms/commit/328b960a0de2097e13654ac752253e9541521ddd
https://github.com/pyca/cryptography/commit/94a50a9731f35405f0357fa5f3b177d46a726ab3
https://github.com/radareorg/radare2/commit/9a310ec866cc802adf7245b76e357ea0f6145e59
https://github.com/radareorg/radare2/commit/64f2493f5840722bf319781340cda83de7e6c8cb
https://github.com/radareorg/radare2/commit/04ea9a5c72de59e03820104113506c4ba3a7108e
https://github.com/radareorg/radare2/commit/fc79e134fa1ebafc00849f129d9a5337c725344c
https://github.com/shantonru/salt/commit/a46c86a987c78e74e87969d8d3b27094e6544b7a
https://github.com/sickrage/sickrage/commit/8156a74a68aea930d1e1047baba8b115c3abfc44
https://github.com/upx/upx/commit/666b3c0f74efaa67858e109df93856a6f26f419c

[saisharan0103]

The Ansible hash fe28767970c8ec62aabe493c46b53a5de1e5fac0 is a good concrete case: it is present in OSV/NVD data but the reported branch containment check cannot find it in the repository.

For a PR, I would start in the commit collection path and add a reachability check so fix commits not contained in any repo branch are skipped.

[ziadhany]

@saisharan0103 I don't think we should skip collecting these commits. The best approach is to collect and report them upstream, then simply flag them in vulnerablecode so they're easy to identify and handle separately.