Any screenshot of what'd happened?
When you select File->Export->Bos FBX (.bos; .fbx), you will see the file pane, and the Bos Fbx Exporter(.bos/.fbx) pane. Within the Bos Fbx Exporter(.bos/.fbx) pane, there is one selection box SDK, which contains only one choice, i.e., FBX SDK 2017. There are 4 checkboxes, they are not checked by default. If you cannot see these, then I guess there is an installation issue. Otherwise, the common issue is, it cannot locate the library or the executable for conversion. If this is the case, there should be an error message in the system console. The system console is off by default, you have to select Window->Toggle System Console to see that.
Try to just export a scene with a cube and check if it works or not. I did not perform many exports with that, if your mesh contains polygon that has more than 4 vertices, it may not work.