It's difficult to begin with as there is literally no hand holding, and some of the early mobs can be very testing, sometimes frustratingly so.
However, once you get into the swing of things and figure out the combat system (it's much more fluid that it first appears to be) you will start to progress much more quickly.
I wouldn't say the difficulty went to 'easy' after a while, more that it ramps up a lot during the beginning stages, almost vertically but then levels out to a more consistent difficulty.
All the above is of course based on the PC version. They may have changed the 360 version somewhat.
Of course, you'll be an old hand at Witcher 2 as you're already on to 3. :wink: