Say you have two polygons (yellow and orange) representing two observations of the same building. It is likely the building has been modified between the observations. How can I generate the smallest possible bounding shape (blue) that encloses both the underlying shapes, and has the same amount of corners as the input polygon with the fewest amount of corners?

The solution needs to be applicable to buildings of all shapes and sizes, but the specific illustrated problem showcases the issue well.

I have looked at using Alpha-shape and Convex hull, but could not produce the desired result.