Or is it just "racism-adjacent"?
But then I saw Jesse Thorn - a nice guy, but white as white can be - use the phrase to describe a black male rap artist, and it made me really uncomfortable.
First of all, there is a misconception that the word "buck" to mean "dollar" has its origins in slavery. That if something was worth "two bucks," that means "I'll trade you two black male slaves for it." This is untrue. The word's actual origins derive from fur trappers, who used deer skin as currency. "Two bucks" meaning, "I'll trade you two buckskins for it."
To confuse the issue, the phrase "young buck" has been taken up by the rap world, as a friend rightly pointed out to me. Among other things, it is the stage name of rap artist David Darnell Brown. But this raises the question for me, whether it's a rap thing like "ridin dirty," or a rap thing like "re-appropriating use of the N-word."
Even Urban Dictionary is - surprisingly - not much help here. All of the entries for "young buck" refer to the rap artist, but the entry for "buck" lists "slang for a young black man" as the third usage, above "male deer" and "slang for money."
Even if the phrase "young buck" doesn't have racist origins, it's definitely a phrase that I would call "racism adjacent." Current usage blurs the line between its meaning, and it can certainly serve as a dog whistle to other racists. At the very least, I think one would be well advised to use caution when trotting this one out. I certainly wouldn't use it to describe a young black man.
Of course, if you say something like "this phrase is racist-ish and should be retired," people get all outraged about it. White people, specifically. But I feel obliged to point out that there are about a million billion word combinations in English that are still available for your use. Phrases that DON'T raise the orange "possibly racist" flag. Why not play it safe, and use one of those instead? Better safe than sorry, right?