java - Transactions don't rollback

Question

I am call two methods, the first one update a table and the next one insert a record in another table. When the second transaction fails the EJB is not doing the rollback of the first transaction.

This is my backing bean:

@ManagedBean
@ViewScoped
public class TransactionTestBean implements Serializable {

    @EJB
    private TransactionTestService service;

    public String loadView() {
        return "/test/transactionTest";
    }

    public void test() {
        try {
            service.updateTest();
        } catch (Exception e) {
        }
    }
}

The EJB interface:

@Local
public interface TransactionTestService {

    void updateTest() throws CustomException;
}

The EJB class:

@Stateless
@TransactionManagement
public class TransactionTestServiceImpl implements TransactionTestService {

    @Resource(mappedName = "java:jboss/datasources/xxxxxDS", shareable = true)
    public DataSource dataSource;

    private TransactionTestDAO dao;

    @PostConstruct
    public void init() {
        dao = new TransactionTestDAOImpl();
    }

    @PreDestroy
    public void destroy() {
        dao = null;
    }

    @Override
    @TransactionAttribute(TransactionAttributeType.REQUIRED)
    public void updateTest() throws CustomException {

        try (Connection connection = dataSource.getConnection()) {
            dao.updateRecord(connection);
            // dao.saveRecord(connection);
        } catch (SQLException exception) {
            throw new CustomException(exception, exception.getMessage());
        }
    }
}

And my custom exception:

@ApplicationException(rollback = true)
public class CustomException extends Exception {

    public CustomException(Throwable cause, String message) {
        super(message, cause);
    }
}

EDITED:

Added the DAO Class:

public class TransactionTestDAOImpl implements TransactionTestDAO {

    @Override
    public void updateRecord(Connection connection) throws CustomException {

        PreparedStatement preparedStatement = null;

        try {
            preparedStatement = connection.prepareStatement("UPDATE table_x SET field_x = ? WHERE field_y = 1");
            preparedStatement.setInt(1, 1);
            preparedStatement.executeUpdate();
        } catch (Exception exception) {
            throw new CustomException(exception, exception.getMessage());
        } finally {
            if (preparedStatement != null) {
                try {
                    preparedStatement.close();
                } catch (SQLException sqlException) {
                }
            }
        }
    }
}

And the DAO Interface:

public interface TransactionTestDAO {

    void updateRecord(Connection connection) throws CustomException;
}

Answer 1

The key issue in this case was bad default in datasources in some JBoss versions. Original code was fine and was working correctly in other application servers (WebSphere App Server and lightweight WebSphere Liberty).

Datasources created in JBoss are not JTA - in admin console the Use JTA setting is unchecked and in xml related setting is <datasource jta="false" .... Changing this setting to true fixed the problem. (JohnB, you wrote that defining xa-datasource fixed that, but since I didn't see your original xml with datasource definition, I believe that during changing datasource you've also change this flawed jta="false" setting). It will work for non xa-datasources also, as Grzesiek tested.

This is a very bad default, since it causes transactions not to be managed by container and causes flawed transaction behavior in connections got in EJB components.

Big thanks to Grzesiek D. who helped me in diagnosing this issue.