Basic question about Signals

clk_50mhz <= not clk_50mhz after 20 ns /2 ;

Here the clk starts and 'U', and the NOT of 'U' is also 'U'. So yes it needs to be initialised.

There is a handy workaround so you can do it without initialisation, if you dont mind using a process (but it means you can add other handy stuff to the testbench):

clk_gen_proc : process
begin
  if clk /= '1' then clk   <= '1';
  else               clk   <= '0';
  end if;
  wait for CLK_PERIOD/2;
  -- Timeout Control. Stops the testbench after G_TIMEOUT clocks without a "normal" finish
  if NOW  >= G_TIMEOUT*CLK_PERIOD then
    report "Simulation ended due to TIMEOUT" severity FAILURE;
    wait;
  end if;
end process;