no it's not silly. Belgium (unlike Germany) represents the union of roman and germanic cultures, exactly like Charlemagne. So even if Charlemagne was born in Germany (or not), he represents far more a Belgian than a German ideal.
because germany invaded belgium twice, imposed flamenpolitik and destroyed that sacred pact (like Saxons taking revenge on Charlemagne) and now we live with the consequences of that...