Damn, didn't even consider the front page stuff. and that was so close to working as required.
Wonder if some of these layout options should be in category settings instead of individual articles. At least then you could have articles that behave as they should on front page, and others in a category where you don't want all this stuff showing. I know each category has a Custom layout setting, but I've never used it, and not even sure if that works properly now.
It's a tricky one to be sure.