Remove an item from the list by item reference. Updates both id_map and fid_map.
The SessionBase item to remove