During his visit Mr Jones was joined by Llanelli AM Lee Waters and Plaid Cymru’s Simon Thomas AM.
This comes following a reported rise in xenophobic comments and abuse in Swansea following the EU vote.
Chairman of the association Jeff Hopkins told the First Minister: “The atmosphere has been uneasy due to the vote.
“Some people thought they were going to send them back. People have been worried.”
Mr Waters said that the vote had been “unsettling for everyone”.
While project manager Halina Ashley said there had been an incident in Trostre, however it was unclear of this had been racially-motivated.
Ahead of the visit the First Minister of Wales said: “I am shocked and appalled by the increase in racial tensions and verbal abuse towards people from ethnic minority communities during the past week. We will not stand for any form of racism in Wales and we will tackle this unacceptable behaviour head on.
“While I have been disgusted by the racial abuse endured by some, I have also been heartened by the public outpouring of support and gratitude for ethnic communities living in Wales. I was immensely proud to hear about the note of thanks pinned anonymously on the door of Llanelli’s Polish Welsh Association. I sincerely hope all other communities follow this wonderful example.”
Mr Jones then took a further tour of the centre, which supports Polish members of the community with advice and translation services.
Project manager Ms Ashley said: “We realise Polish people need support in every aspect to make integration as smooth as possible.
“We go to the schools to help translate for parents, we also work with the courts and police to help where we can. We work with everyone, anyone who needs us.”
She then asked the First Minister and AMs if they would meet with the Polish community to continue promoting understanding and strong relations following the vote for Brexit.
During the visit Mr Hopkins said their support reinforces to Polish people the backing they have got from the Welsh Government.
He said: “It seems there has been more anxiety since the vote. Particularly when you hear inflammatory statements from politicians.
“But this visit shows they have support.”