Dictionary of American idioms

BACK DOWN (FROM SOMEONE OR SOMETHING)

back down (from someone or something): translation

back down (fromsomeone or something)
to yield to a person or a thing; to fail to carry through on a threat. •

Jane backed down from her position on the budget.

It's probably better to back down from this situation; looks like you are at a disadvantage.