Does my common-law ex have the right to take back something that they bought for me?

Whether it is a car, or any other item of property, in law this will depend entirely on the manner and circumstances in which the item was given.

In some situations, it will be clear that your Ex intended to make an outright gift of the car to you. The surrounding facts at the time of the giving over will establish whether this is a reasonable assumption.

In the case of a car it might involve formal things like registering the car in your name (so that you are listed as the owner, not just the driver).  It may also involve more informal gestures at the time of the gift, like making a big display of giving it to you (e.g. putting a big bow on the car, and presenting it to you on your birthday).

In other cases, the circumstances will reasonably suggest that the item was intended for your use in limited or narrow circumstances, but that there was never any intent on the part of your Ex to transfer ownership.  In that case, you would not be entitled to keep the car when you decide to split.

In the circumstances you describe, it appears that the car was essentially “loaned” to you for those times when you were watching your Ex’s child.  It is likely that your designation as driver was done merely to facilitate that goal — it is unlikely that this step alone would be enough for a court to decide that your Ex intended a gift.

If your Ex did not intend to give you the car as a gift (and it seems like this is already a contentious point), then if you take it anyway — whether for a short or long period — there could be serious repercussions if he decides to bring either a criminal charges (for theft) or a civil claim (for conversion), among other possibilities.

However, it is difficult to make an assessment without the fuller story.  The factual context, plus other indications of what your Ex intended at the time, will determine the legal status of the car in your hands.