When using an OpenDyslexic Chrome extension, the pages will all appear in OpenDyslexic. The official extension accounts for icon fonts and has an on/off switch. Some others may not provide either.

In either case, if you need to remove an extension, follow the instructions here: https://support.google.com/chrome_webstore/answer/2664769?hl=en