Nevertheless, it ended up in the canon of Scripture as the word of God. To say it shouldn't be, then begs the question: What else was canonized that we cannot rely on? IMO we have to trust God as the architect and overseer of the canonization, and let it rest there. I have read the history of how James became a part of the canon, but it was a long time ago, and I don't remember a thing I read about it.
I think it is helpful to recognize what type of literature James is, and even though it is a letter, its contents smack strongly of the same type of phrasing and content as the wisdom literature of the OT. "It is wise to do this. It is unwise to do that." There are doctrinal statements in it, but that does not appear to be his purpose in writing it and to whom he wrote it. I think that we can be certain that James agreed with Paul on justification by faith alone. James is not writing about justification. He is writing about the evidence of salvation (faith). If a profession of faith is made but there are no works (and that includes more that what the Catholics consider good works. It is the gradual transformation of sanctification), they are still dead in their trespasses and sins. They have not been brought to life by the Spirit. If the professed faith is real, it will bear the fruit of good works.
Luther was in a particular position when he began his work of reforming the Catholic church. And a particular thing that he found in scripture, lit that flame. And it was salvation through faith alone. It was a bit of an obsession. Just as some people on forums are only interested in specific areas such as end times. So his reaction to the book of James might have been a knee jerk emotional reaction. He was brilliant. He was being led by God to do the work that he was doing. But he was also, still human.
James is not contradicting Paul's teaching. If it were, it would not be in our Bible which is the word of God. If it looks like a contradiction, it is up to us to find out why, from what is clear on the subject in the rest of scripture, (in this case starting with what it is not) why James said what he said.