Easy Ease is always a curve

The value graph for time remapping presents a line that reflects time increasing vertically. Constant time is a straight line graph starting from the bottom and going to the top. If you start out in slow motion and increase to fast motion you will get a curve that describes an arc. The steep parts of the curve are where time moves faster, the shallow parts of the curve are where time moves more slowly.

The Speed Graph does not show you speed, it shows acceleration. If you edit the speed graph for time remapping you see the acceleration values. A horizontal line is a constant speed, a vertical line is an instant change to a new speed. Speeding up then slowing down in creates a speed graph that looks like an upside-down U.  Visualizing what is happing in the graph editor takes a bit of practice. Maybe this will help:

In the second screenshot, the shot starts out ahead of the real-time then slows down as the speed graph shallows out, then runs at nearly realtime but a little later than the realtime, then slowly speeds up until the last second slows down to very slow motion. I usually put markers where I want certain events to happen in the timeline, then adjust the speed graph using Auto Bezier instead of Easy Ease to move the frames to where they need to be to match the timing of the shot. As long as you have the pen tool or the selection tool (g or v) selected you should be able to enable easy ease or auto bezier in the graph editor and drag out handles.