DataFusionBlock enhancements to extract and fuse optical flow features into the model - #17
Open
yusfcicek wants to merge 3 commits into
Open
DataFusionBlock enhancements to extract and fuse optical flow features into the model#17yusfcicek wants to merge 3 commits into
yusfcicek wants to merge 3 commits into
Conversation
yusfcicek
commented
Apr 9, 2024
yusfcicek
commented
- In training and inference time extracts optical flow features in multi frames
- Multiple frames are taken as a ring buffer, then per-frame optical flow features are extracted
- Frames and optical flow features are fused by DataFusionBlock
- Configuration improvements were made according to the number of datasets
- Fixed bugs in torch functions in Python 3.9 and above
…lti frames * Multiple frames are taken as a ring buffer, then per-frame optical flow features are extracted * Frames and optical flow features are fused by DataFusionBlock * Configuration improvements were made according to the number of datasets * Fixed bugs in torch functions in Python 3.9 and above
Prepend a fork note to the README explaining what this branch adds on top of SysCV/MaskFreeVIS, with upstream attribution kept intact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Tue2dg5QrW9Jwjcsqe94n
The two frames handed to calcOpticalFlowFarneback were converted to grayscale with different colour orders -- COLOR_RGB2GRAY for the previous frame and COLOR_BGR2GRAY for the next. The two conversions apply the luminance weights to different channels, so consecutive frames landed in different grayscale spaces and the estimated flow carried a systematic error. Both now use COLOR_RGB2GRAY, matching the documented RGB input. The same docstring named Lucas-Kanade while the code calls the Farneback estimator, and the docstrings in build.py and base.py were copied from detectron2's backbone equivalents and described building a backbone. Also renames fusioned_* to fused_* and concanated_feature to concatenated_feature, and fixes the wording in the OpticalFlowFusionBlock docstring. All call sites are in this repository, so the keyword argument rename is complete. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZLNft11UJQfqcRXhcHnML
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.