CL_INVALID_BINARY for larger design

I can think of two reason for CL_INVALID_BINARY:

1. You are compiling against an incorrect BSP or board.

2. The function in your host code that reads the kernel file is somehow corrupting it.

If your problem depends on the size of the binary file, it is quite likely that the cause is the kernel reading function. Are you properly allocating memory using malloc to read the kernel file? Have you made sure that the memory buffer for the kernel is large enough?