I'm just stating the facts, holding colonial rule against them when they had minimal political impact for most of it is bizarre to me, if anything most of the empire building happened after their diminished role with overwhelming public support , hell some of the more egregiously acts happened under governments which by today's standards will be considered relatively liberal.
Anyway them being apolitical has probably led to them lasting so long, making enemies out of half your population without any effective executive powers is daft, still I think if any criticism was to be leveled against them it has to be about them benefiting from public funds and such.