Regression: selection in Wonderbrush is not transparent anymore

[pulkomandy]

The selection rectangle in wonderbrush is now opaque white, making wonderbrush essentially unusable. This used to work fine in previous Haiku versions but I have not needed Wonderbrush in a while

[pulkomandy]

Drawing is done using ConstrainClippingRegion and FillRect. As far as I can see it should only draw a stripped border around the selection, but the inside ends up being filled in white.

Possibly hrev55510 ? Can't test because building haiku on haiku 32bit is not possible currently (gcc11/gcc8 mismatches)

[humdinger]

I've seen that for a very long time. In fact, I cannot remember the selected-object-rect not being white and I always thought that intentional though a poor choice...

[waddlesplash]

GCC11/8 mismatches now resolved.

[madmax]

Looks like a problem in the subpixel renderer or nearby. Change antialiasing to grayscale and you get just the border in selections and the navigation view.

Clipping doesn't seem to be involved: can be reproduced with just StrokeRect (or StrokeEllipse or...) called with a pattern that is not B_SOLID_HIGH nor B_SOLID_LOW. The end result seems to be the shape is filled with low color and then stroked with pattern.

[pulkomandy]

There is a problem with StrokeRect when using a pattern. This change to disable the pattern results in Wonderbrush drawing just the outline (as expected):

diff --git a/WonderBrush/src/canvas_view_states/PickObjectsState.cpp b/WonderBrush/src/canvas_view_states/PickObjectsState.cpp
index 968b958..62f1d14 100644
--- a/WonderBrush/src/canvas_view_states/PickObjectsState.cpp
+++ b/WonderBrush/src/canvas_view_states/PickObjectsState.cpp
@@ -111,10 +111,10 @@ PickObjectsState::Draw(BView* into, BRect updateRect)
                into->SetHighColor(kBlack);
                into->SetLowColor(kWhite);
                BRect box = fCanvasView->ConvertFromCanvas(fBoundingBox);
-               into->StrokeRect(box, kDotted);
+               into->StrokeRect(box/*, kDotted*/);
                if (box.Width() >= 2.0 && box.Height() >= 2.0) {
                        box.InsetBy(1.0, 1.0);
-                       into->StrokeRect(box, kDotted);
+                       into->StrokeRect(box/*, kDotted*/);
                }
        }
 }

What's strange is, I tried to disable B_SUBPIXEL_PRECISE in WonderBrush and it does not make any difference. A completely different codepath is used in app_server.

So, this is a problem in code that's shared between the two cases.

Switching to B_OP_OVER or B_OP_ADD works (but looks a bit different), so it's a problem only with B_OP_COPY mode.

[pulkomandy]

Proposed fix: ​https://review.haiku-os.org/c/haiku/+/4836

[X512]

Replying to pulkomandy:

Proposed fix: ​https://review.haiku-os.org/c/haiku/+/4836

How it worked before?

[pulkomandy]

As humdinger said, this was broken for a really long time and maybe it never worked.

[waddlesplash]

Fix merged in hrev55749.