The initial value of state is false. So, after you do the first click, you set state to !state, which makes it true (because !false equals true). On the second click, it will become false because state was true and setting it to !state made it false (!true equals false). The exclamation mark (!) is read as not.

By the way, I recommend setting the state like this:

setState(prev => !prev);

Hope this helps.

By moving the State into the main ToggleMenu, you can have this component maintain the visibility of the Menu.

class ToggleMenu extends React.Component {
    constructor(props) {
        super(props);

        this.state = {
            visible: false  
        };

        this.toggleMenu = this.toggleMenu.bind(this);
    }

    toggleMenu() {
        this.setState({visible: !this.state.visible})
    }

    render() {
        return (
            <div>
                <button onClick={this.toggleMenu}>Show Right Menu!</button>
                {this.state.visible && <Menu alignment="right">
                <MenuItem hash="first-page">First Page</MenuItem>
                <MenuItem hash="second-page">Second Page</MenuItem>
                <MenuItem hash="third-page">Third Page</MenuItem>
                </Menu>}
            </div>
        );
    }
}

This allowed me to change your menu into a Stateless Component:

const Menu = ({alignment, children}) => (
    <div className="menu">
        <div className={alignment}>{children}</div>
    </div>
);

I have created a webpackbin here (now with animation): https://www.webpackbin.com/bins/-Klh1VM-n4RDCkEbkK67

For transitions and animation, I recommend you look at https://github.com/reactjs/react-transition-group