16.5% APR is a touch under 1.5% monthly (12th root of 116.5).
Assuming lump-sum repayment at the end of the period...
Your friend borrowed $3700, held it for 11 months, then repaid. $3700*(116.5^1/12)^11) comes out to total cost of $4256.
(Good illustration of why credit card interest rates should be avoided whenever possible -- they're huge and they add up fast.)
Actually the cost to you may have been greater than that, since as long as this stayed in the card you were paying interest immediately on all your own charges, rather than being able to take advantage if the grace period. But since this is a friendly transaction, and since I'm not willing to go through your card statements for the past 11 months, I suggest that you just forgive those losses and treat them as a gift to the friend.
Credit card interest is generally compounded daily. So if you want to compute this accurately, you need to figure out how many days you are being charged interest. There is usually a grace period for new purchases, where you aren't charged any interest until your first bill; however, the grace period only applies if you pay your statement in full. Since you were already carrying a balance on your card before this purchase, there was no grace period. The number of days would be from the date of purchase to the date of payment.
The formula for compound interest is:
where P is the principal ($3700), r is the daily interest rate (16.5%/365 = 0.0004521), and n is the number of days you are charging interest.
If the number of days is 330 (about 11 months), the interest would be $595.11.
This assumes that he didn't pay anything to you until the one payment after 11 months.
In the comments below, the OP said:
by having that amount on the card, the minimum payment was about $200 a month, and I made little headway in ever reducing the debt. Had that amount not been on my card, I would have been paying the same amount and making headway on the debt. Having that charge on my card cost me money, that is for certain. I'm trying to work out how much.
Your $200 monthly payment covered all of your interest and your friend's interest and helped pay down your debt. Yes, if you didn't have this purchase on your card and were still paying the $200 each month, you would have reduced your debt more. However, your friend did give you $500 in early payments, which helped you make your credit card payment.
Each of these monthly payments that you made includes a small amount that goes toward reducing your debt (principal). By charging your friend the $595 calculated above, you are crediting all of these principal payments for your own debt (as you should, since your friend wasn't really making any payments). However, you could have paid more toward the principal, and chose not to. It's not really your friend's fault that you didn't pay more.
However, I think I see where you are coming from. You are saying that because you had to pay his portion of the interest over these months, you missed out on the extra debt reduction and had to pay additional interest on your portion. For example, in the first month, $50.51 of your payment was for his interest. If that $50.51 had gone toward your principal instead, you would have saved $8.12 in interest over the next 11 months. Each of the following months you would have less savings, because there are fewer months left for you to save interest over. I put together a spreadsheet that shows the "lost savings" each month:
Based on this premise, having to pay your friend's interest each month means you lost out on $49.84 that you could have saved if you had instead been paying down your principal.
However, we need to look at the other side of this in Column B. You'll notice that your friend's interest charge goes up each month, because he hasn't been paying on it. However, you have been paying it each month, which means that the interest you have been paying on it has essentially been a flat $50.51 each month. By charging your friend the increasing amounts shown in column B, you are already essentially collecting the "lost savings" in column D. If you charge him both, you are really charging him twice, in my opinion.
Others have addressed the math so I'll just focus on the philosophy...
I want to be sure not to charge him for any interest accrued for the debt I already had on the card, and I am not sure how to separate the two.
Why? It seems like you're doing your friend a [gigantic] favor, letting him store some debt in your name and you're actually quite lucky to have been paid back the $3,700. I'm not sure why your interest rate on the credit card matters. You assumed a lot of risk in this transaction; risk that you should be compensated for.
I understand the you probably think it would be "unfair" to charge a rate higher than what you were charged, but you should spend a minute searching on here for the horror stories that come after favors like this fall apart. "I cosigned a car but the person moved across the country and stopped paying the loan," etc.
Fact of the matter is, as far as the bank is concerned, that debt is yours. If you charge your friend 17% or 20% you're still severely under representing your risk in the situation. Using 17% (just +0.5% to your APR) to this calculation over 11 months only adds about $17 to the total which isn't even enough to file a small claims case if your friend stops paying you - never mind collection costs, time, and the fact that interest will continue accruing on the debt that was of no benefit to you.