Spaces:
Sleeping
Weird artifacts with long video
Hey, I'd like to express my appreciation for your exceptional work here. Thank you!
I've been experimenting with your demo and it performs remarkably well with shorter videos, i.e. when the number of keyframes is < 10. However, I've noticed that when the keyframes exceed approximately 15-20, visual artifacts begin to appear and these issues seem to accumulate over time, detracting from the overall quality. I'm curious as to what might be causing this and if there is a potential solution.
Config:
color_preserve=True
cross_start=0
cross_end=1
style_update_freq=1
warp_start=0
warp_end=0.1
mask_start=0.5
mask_end=0.8
ada_start=0.8
ada_end=1
mask_strength=0.5
inner_strength=0.9
smooth_boundary=True
use_constraints = [
'shape-aware fusion', 'pixel-aware fusion', 'color-aware AdaIN'
]
First keyframe:
10th keyframe, still awesome:
15th keyframe, artifacts start appearing (look at the eyes and hair):
Later keyframe (probably 70th), artifacts accumulation:
This is because the error accumulation during the cross-frame fusion.
You can try one of more of the following settings to alleviate the error accumulation:
- increase
Cross-frame attention update frequency - increase
Key frame frequency - decrease
Pixel-aware fusion strength - decrease
Pixel-aware fusion detail level