Consider switching to the Time series visualization type. It is a more capable and performant version of this panel.
Migrate
Some features like colored time regions and negative transforms are not supported in the new panel yet.