Getting changes from collection

Good morning :)

Usually to update some entities in database I use service that must be calculate changes from new and old object and update changes in database(using DAO, for example). And the common task is update collections. I wrote simple class that calculate changes from old and new collections and create three collections: to delete, to update and to create. You can see class below and some description for it

package com.blogspot.jajatips.utils;

import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.collections.Predicate;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import java.util.*;

public class ChangesCollections<T> {
 private static class CollectionFactory<T> {
  private static final Log log = LogFactory.getLog(CollectionFactory.class);
  private Class<? extends Collection> classCollection = ArrayList.class;

  private CollectionFactory(Class<? extends Collection> classCollection) {
   this.classCollection = classCollection;
  }

  @SuppressWarnings("unchecked")
  public Collection<T> getNewCollection() {
   try {
    return classCollection.newInstance();
   } catch (Exception e) {
    log.warn("Can not create collection for class [" + classCollection.getName() + "], create java.util.ArrayList");
    return new ArrayList<T>();
   }
  }
 }

 private Collection<T> deleteCollection;
 private Collection<T> addCollection;
 private Collection<T> updateCollection;

 public ChangesCollections(Collection<T> oldCollection, Collection<T> newCollection) {
  CollectionFactory<T> factory = new CollectionFactory<T>(oldCollection.getClass());

  this.deleteCollection = factory.getNewCollection();
  this.addCollection = factory.getNewCollection();
  this.updateCollection = factory.getNewCollection();

  addCollection.addAll(newCollection);
  deleteCollection.addAll(oldCollection);
  Iterator<T> addIt = addCollection.iterator();
  while (addIt.hasNext()) {
   T addObject = addIt.next();
   if (contains(oldCollection, addObject)) {
    updateCollection.add(addObject);
    deleteCollection.remove(addObject);
    addIt.remove();
   }
  }
 }

 public Collection<T> getDeleteCollection() {
  return deleteCollection;
 }

 public Collection<T> getAddCollection() {
  return addCollection;
 }

 public Collection<T> getUpdateCollection() {
  return updateCollection;
 }

 private boolean contains(Collection<T> collection, T object) {
  if (object.getClass().isAssignableFrom(Predicate.class)) {
   return CollectionUtils.exists(collection, (Predicate) object);
  }

  return collection.contains(object);
 }
}

First of all, to build this code you need next libraries: commons-collections and commons-logging. Also, I like generics and use it in all my code, because I like compile errors more than runtime errors :).
Now, about the code. To create collections, I use inner class CollectionFactory, that try to create class of collection same as original collection, if it can not do this, it create standard java.util.List. As you can see, code is simple. All that it done is iterate over collections and put elements in other three collections.

I hope that this class helps you in your projects :)

Write compressing rolling file appender for log4j

UPD: One good man leave a good comment :)

I think this is now supported in the standard;

http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html


Good day!

On the last week, we gave our software to support specialists. They installed the program to production servers. And over three thousand users began to work with it. At the on of that week, I didn't hear nothing about our product. This is good the good sign! :) But, we have one little problem, log file that wrote program had size little more 1Gb!!! This is not very good result :) Server's administrator sent me this log file. I analyzed it. Unfortunately, all information in it was very useful and didn't find anything that I can remove from log.

For logging, we use log4j. If you never use log4j, I recommend to read official document. Log4j have interface Appender. When you want log something, you call some code, looks like this:

...
import org.apache.log4j.Logger;

public class YourClass {
  private static final Logger log = Logger.getLogger(YourClass.class)
  ...
  public void someMethod() {
    log.info("Call someMethod()");
  }
  ...
}

Log4j pass all logging events through appenders, that you configure. By default, you have many appenders, that can send log over SMTP, JMS and do other interesting things. But, this list has no appenders that can periodically compress log file. To solve this problem, I write my own appender, that can periodically compress current log file. Because, I need compress only files, I inherit my appender from FileAppender and override subAppend() method. I don't speak more and simple show you result code:)

import org.apache.log4j.FileAppender;
import org.apache.log4j.helpers.LogLog;
import org.apache.log4j.spi.LoggingEvent;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.FileOutputStream;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.GregorianCalendar;
import java.util.zip.GZIPOutputStream;

/**
 * Appender that periodically compress current log.
 * You need only pass one setting - period after that log should compress.
 * Available periods are MINUTLY, HOURLY, DAILY, WEEKLY, MONTHLY.
 *
 * @author Pokidov.Dmitry
 *         Date: 27.08.2009
 */
public class CompressingRollingFileAppender extends FileAppender {
 enum Period {MINUTLY, HOURLY, DAILY, WEEKLY, MONTHLY}

 private GregorianCalendar rollDate;
 private Period period = Period.HOURLY;
 private SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd-HH-mm");

 @Override
 public void activateOptions() {
  super.activateOptions();
  setAppend(false);
 }

 /**
  * Period for rolling. By default - {@code HOURLY}
  * @param period new period. Available values: MINUTLY, HOURLY, DAILY, WEEKLY, MONTHLY.
  */
 public void setPeriod(String period) {
  try {
   this.period = Period.valueOf(period);
  } catch (IllegalArgumentException e) {
   LogLog.warn("No period with name [" + period + "]", e);
  }

  LogLog.debug("Set roll period to [" + this.period + "]");
 }

 @Override
 public void setAppend(boolean flag) {
  LogLog.warn("Compression rolling appender doesn't support append option(always set to false)");
 }

 protected void rollOver() {
  String currentFileName = getFile();
  File currentFile = new File(currentFileName);
  File zipFile = new File(currentFileName + "." + dateFormat.format(new Date()) + ".gz");
  try {
   /*
     actually, I use helper that read all bytes, using NIO.
     see http://jajatips.blogspot.com/2008/11/reading-from-inputstream.html
   */
   FileInputStream stream = new FileInputStream(currentFile);
   byte[] content = new byte[](stream.available());
   stream.read(content);

   GZIPOutputStream gzipOutputStream = new GZIPOutputStream(new FileOutputStream(zipFile));
   gzipOutputStream.write(content, 0, content.length);
   gzipOutputStream.finish();
   gzipOutputStream.flush();
   gzipOutputStream.close();

   setFile(currentFileName, false, getBufferedIO(), getBufferSize());
   rollDate.add(getDateFieldByPeriod(), 1);
  } catch (IOException e) {
   LogLog.error("Error compress current log file [" + currentFile + "] to zip file [" + zipFile + "]", e);
  }
 }

 @Override
 protected void subAppend(LoggingEvent event) {
  if (rollDate == null) {
   rollDate = new GregorianCalendar();
   rollDate.add(getDateFieldByPeriod(), 1);
  }

  GregorianCalendar now = new GregorianCalendar();
  if (rollDate.before(now)) {
   rollOver();
  }

  super.subAppend(event);
 }

 private int getDateFieldByPeriod() {
  switch (period) {
   case MINUTLY:   return GregorianCalendar.MINUTE;
   case HOURLY:    return GregorianCalendar.HOUR;
   case DAILY:     return GregorianCalendar.DAY_OF_MONTH;
   case WEEKLY:    return GregorianCalendar.WEEK_OF_MONTH;
   case MONTHLY:   return GregorianCalendar.MONTH;
   default: return GregorianCalendar.MINUTE;
  }
 }
}

Also, I wrote test, because I don't want situation when error in logging subsystem cause crash in the application :)

import junit.framework.Assert;
import junit.framework.TestCase;
import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;

import java.io.File;
import java.io.StringReader;
import java.io.IOException;
import java.util.Properties;

/**
 * @author Pokidov.Dmitry
 *         Date: 28.08.2009
 */
public class CompressingRollingFileAppenderTest extends TestCase {
 private static final String LOG_DIR="log";

 private static class LogRun implements Runnable {
  private Logger log = Logger.getLogger(LogRun.class);

  @Override
  public void run() {
   for (int i = 0; i < 150; i++) {
    log.info("Log " + i);
    try {
     Thread.sleep(1000);
    } catch (InterruptedException e) {
     log.error("Interrupt", e);
    }
   }
  }
 }

 /**
  * Create 2 threads and create some log in it.
  */
 public void testAppender() {
  File dirLog = new File(LOG_DIR);
  try {
   for (File f : dirLog.listFiles()) {
    if (!f.delete()) {
     System.err.println("Cann't delete old log file [" + f.getName() + "]");
     Assert.fail("Cann't delete old log file [" + f.getName() + "]");
    }
   }

   PropertyConfigurator.configure(getLogProperties());

   Thread logThread1 = new Thread(new LogRun());
   Thread logThread2 = new Thread(new LogRun());

   logThread1.start();
   logThread2.start();

   logThread1.join();
   logThread2.join();
  } catch (Exception e) {
   e.printStackTrace();
   Assert.fail("Error while execute test: " + e.getMessage());
  }

  Assert.assertEquals(3, dirLog.list().length);
 }

 private Properties getLogProperties() throws IOException {
  Properties result = new Properties();
  String cfg = "log4j.debug=TRUE\n" +
               "log4j.rootLogger=INFO, main\n" +
               "\n" +
               "log4j.appender.main=com.otr.security.server.utils.CompressingRollingFileAppender\n" +
               "log4j.appender.main.Period=MINUTLY\n" +
               "log4j.appender.main.Append=true\n" +
               "log4j.appender.main.layout=org.apache.log4j.PatternLayout\n" +
               "log4j.appender.main.layout.ConversionPattern=[%d] %5p [%t] (%F:%L) - %m%n\n" +
               "log4j.appender.main.File=" + LOG_DIR + "/test.log";

  result.load(new StringReader(cfg));

  return result;
 }
}

And the production log4j.properties:

log4j.debug=TRUE
log4j.rootLogger=INFO, main

log4j.logger.com.mchange=ERROR
log4j.logger.org.springframework=ERROR

log4j.appender.main=server.utils.CompressingRollingFileAppender
log4j.appender.main.Period=DAILY
log4j.appender.main.File=log/server.log
log4j.appender.main.layout=org.apache.log4j.PatternLayout
log4j.appender.main.layout.ConversionPattern=[%d] %5p [%t] (%F:%L) - %m%n

Using inner classes

Hi all! In this post I want to tell you about inner classes. Very often, we create static inner classes that contains information about main class. For example:

 public class Person { 
   private String name;
   private String occupation;
   public static class PersonInfo {
     private String info;
     public PersonInfo(String name, String occupation) {
       info = “His name is ” + name + “ . He is a ” + occupation;
   }
   public String getInfo() {
     return info;
   }
 }
 public Person(String name){
   this.name = name;
 }
 public void setOccupation(String occupation){
   this.occupation = occupation;
 }
 public String getOccupation() {
   return occupation;
 }
 public PersonInfo getInfo() {
   return new PersonInfo(name, occupation);
 }
 } 

This is a good example of using inner class. It demonstrates one of the main OOP conception – encapsulation. But it has one problem. If we put this object in some model, model don't know anything about Person and if Person object will be change, model won't know about this. And now I want to show powerful feature of inner class in java – this.outer:

public class Person {

  private String name;

  private String occupation;

  public class PersonInfo {

    public PersonInfo() {}

    public String getInfo() {

      return “His name is ” + this.Person.name +

                      “ . He is a ” + this.Person.occupation;

    }

}

public Person(String name){

  this.name = name;

}

public void setOccupation(String occupation){

  this.occupation = occupation;

}

public String getOccupation() {

  return occupation;

}

public PersonInfo getInfo() {

  return new PersonInfo();

}

}

Now, PersonInfo will return actually information about Person at anytime. And I shouldn't reload model that use it when Person object change. I can use feature this.superClass in anonymous class, too. Notice, that PersonInfo is not static. Actually, in first simple example PersonInfo is standalone class. In the second example, PersonInfo is in Person object.

Debug JNLP application

Hi all!

I want to remote debug in jnlp application. This problem solved is very simple. I set environment variable JAVAWS_VM_ARGS to value: -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5006

After this, I create new run point in Intelli J IDEA. Type of run point is Remote debug. In run point set set host=localhost and port=5006(or other, ports in JAVAWS_VM_ARGS and run point must be equals).

All is ok, run jnlp and created run point.

Using reflections for test equals.

Problem: test for equals two objects of the same class by all fields(deep equals).
For solve this problem I use reflection API from JDK. Package that store classes for reflection is java.lang.reflect. I wrote simple helper class with static methods.

You can find the complete realization at the and of the post.

First of all we should describe algorithm:
Input: two object that we need compare.
Steps:
1. Check objects references

obj1 == obj2

2.Check that names of classes for this objects are identically

c1.getName().compareTo(c2.getName()) == 0

3.While class is not java.lang.Object do steps from 4 to 8

while (!c1.getName().equals(Object.class.getName()))  {...}

4.Go through all fields in class

for (Field f1 : c1.getDeclaredFields())

5.Ignore inner classes, because they have cross references and we have infinity loop.

if (!f1.getName().contains("this$")) {...}

6.Get the same field from second class.

Field f2 = c2.getDeclaredField(f1.getName());

7.Turn off access checking(we want equals all:))

f1.setAccessible(true);
f2.setAccessible(true);

8.Call method that do test objects of these fields and if its result is fail return false.

if (!doObjectEquals(f1.get(obj1), f2.get(obj2))) {
    return false;
}

9.When process all fields, get the super classes of our objects, cast objects to superclass and go to step 4.

c1 = c1.getSuperclass();
c2 = c2.getSuperclass();
obj1 = c1.cast(obj1);
obj2 = c2.cast(obj2);

Now, we can write method that will be equals two objects. First of all, we should check this objects for nulls. If both objects are null, then we do nothing and return true. If one of this object is null...then we do nothing, too :), but return false. For equals two objects, we should try to find declared method equals(). If the object have this method, then we call it and return it's result, else we should recursively call first method for equals each field in this object. Also in this method we should process arrays and collections. In Java, Collections API have three kind of objects. There are Set, List and Map.  In Map we can compare Set of Map.Entry objects, that returns method Map.entrySet(). Notice, that these methods compare items in array without order.

I write some static helpers for equals collections using reflections.

Also, we should specially equals  fields of enum types.  We can check field using method Field.isEnumConstant(). If field is enum constant, we should simple equals() values .

Finally, helper class looks like this:

import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.*;

/**
 * @author : Pokidov.Dmitry
 * @date: 27.05.2008
 * 
* Class that help to test equals for two object.
 * For test use {@code weakEquals()} method.
 */
public class WeakEqualsHelper {
 private static ArrayList<String> errorLog = new ArrayList<String>();
 private static Map<String, List<String>> fields;

 private enum CheckNullsResult {
  BOTH_NULLS, NOT_NULLS, ONE_NULL;
 }

 /**
  * Recursively compare object by its class and all superclasses.
  * To standalone test collections, arrays and maps use methods: 
  * {@code equalsCollection(), equalsArray(), equalsMap()}
  *
  * @param o1 First object for compare
  * @param o2 Second object for compare
  * @return {@code true}, if objects are equals, else {@code false}
  */
 public static boolean weakEquals(Object o1, Object o2) {
  return weakEquals(o1, o2, null);
 }

 /**
  * Recursively compare object by its class and all superclasses.
  * To standalone test collections, arrays and maps use methods: 
  * {@code equalsCollection(), equalsArray(), equalsMap()}
  *
  * @param o1 First object for compare
  * @param o2 Second object for compare
  * @param notEqualsFields fields that will be ignore. Key is name of class,   Value - list of field's name.
  * @return {@code true}, if objects are equals, else {@code false}
  */
 public static boolean weakEquals(Object o1, Object o2, Map<String, List<String>> notEqualsFields) {
  fields = notEqualsFields;
  boolean ret = doEquals(o1, o2);
  if (!ret) {
   for (String error : errorLog) {
    System.out.println(error);
   }
  }
  return ret;
 }

 public boolean weakEquals(Object o) {
  if (o == null ||
    !(o instanceof WeakEqualsHelper)) {
   return false;
  }
  return o == this || doEquals(this, o);
 }

 private static boolean needEqual(Field f, Map<String, List<String>> fields) {
  if (fields != null) {
   List<String> classFields = fields.get(f.getDeclaringClass().getName());
   return !(classFields != null && classFields.contains(f.getName()));
  }

  return true;
 }

 public static boolean doEquals(Object obj1, Object obj2) {
  if (obj1 == obj2) {
   return true;
  }
  Class c1 = obj1.getClass();
  Class c2 = obj2.getClass();
  if (c1.getName().compareTo(c2.getName()) == 0) {

   while (!c1.getName().equals(Object.class.getName())) {
    for (Field f1 : c1.getDeclaredFields()) {
     if (needEqual(f1, fields)) {
      String errorMessage = "ERROR: Class " + obj1.getClass().getName() + " are not equal in field -->" + f1.getName() + "<--";
      if (!f1.getName().contains("this$")) {
       try {
        Field f2 = c2.getDeclaredField(f1.getName());

        f1.setAccessible(true);
        f2.setAccessible(true);

        errorLog.add("Current field-->" + f1.getName() + "<--");

        if (f1.isEnumConstant()) {
         if (!f1.get(obj1).equals(f2.get(obj2))) {
          return false;
         }
        } else if (!doObjectEquals(f1.get(obj1), f2.get(obj2))) {
         errorLog.add(errorMessage + "[" + (f1.get(obj1) != null && f1.get(obj1).equals("null") ? "NULL" : f1.get(obj1)) + ":" + (f2.get(obj2) != null && f2.get(obj2).equals("null") ? "NULL" : f2.get(obj2)) + "]");
         return false;
        }
       } catch (NoSuchFieldException nsme) {
        errorLog.add("ERROR: No such field " + f1.getName());
        return false;
       } catch (IllegalAccessException e) {
        e.printStackTrace();
        return false;
       }
      } else {
       errorLog.add("INFO: ignore field -->" + f1.getName() + "<--");
      }
     } else {
      System.out.println("INFO: ignore field -->" + f1.getName() + "<--");
     }
    }

    c1 = c1.getSuperclass();
    c2 = c2.getSuperclass();
    obj1 = c1.cast(obj1);
    obj2 = c2.cast(obj2);
   }
  } else {
   errorLog.add("Class names are not equals in field[" + c1.getName() + "," + c2.getName() + "]");
   return false;
  }
  return true;
 }

 private static boolean doObjectEquals(Object o1, Object o2) {
  if (checkNulls(o1, o2) == CheckNullsResult.NOT_NULLS) {
   if (o1 instanceof Map.Entry) {
    Map.Entry e1 = (Map.Entry) o1;
    Map.Entry e2 = (Map.Entry) o2;
    return doObjectEquals(e1.getKey(), e2.getKey()) &&
      doObjectEquals(e1.getValue(), e2.getValue());
   } else if (o1 instanceof Map) {
    return equalsMap((Map) o1, (Map) o2);
   } else if (o1 instanceof Collection) {
    return equalsCollection((Collection) o1, (Collection) o2);
   } else if (o1.getClass().isArray()) {
    return equalsArray((Object[]) o1, (Object[]) o2);
   } else if (!o1.getClass().getName().equals(Object.class.getName())) {
    if (hasEquals(o1.getClass())) {
     if (!o1.equals(o2)) {
      if (o1 instanceof String &&
        o2 instanceof String) {
       String s1 = ((String) o1).replaceAll("[\n\t\r ]", "");
       String s2 = ((String) o2).replaceAll("[\n\t\r ]", "");
       if (!s1.equals(s2)) {
        return false;
       } else {
        System.out.println("WARNING: Strings are not equals with escape symbols and whitespaces");
        return true;
       }
      }
      System.out.println("Equals return false");
      return false;
     }

     return true;
    } else {
     return doEquals(o1, o2);
    }
   }
   System.out.println("UNKNOWN TYPE: " + o1.getClass().getName());
  }

  return equalsForNull(o1, o2);
 }

 private static CheckNullsResult checkNulls(Object object, Object otherObject) {
  if (object == null && otherObject == null) {
   return CheckNullsResult.BOTH_NULLS;
  }
  if (object != null && otherObject != null) {
   return CheckNullsResult.NOT_NULLS;
  }

  return CheckNullsResult.ONE_NULL;
 }

 private static boolean equalsForNull(Object object, Object otherObject) {
  return object == null && otherObject == null;
 }

 public static boolean equalsMap(Map map, Map otherMap) throws NullPointerException {
  return equalsCollection(map.entrySet(), otherMap.entrySet());
 }

 public static boolean equalsCollection(Collection collection, Collection otherCollection)
   throws NullPointerException {
  return equalsArray(collection.toArray(), otherCollection.toArray());
 }

 public static boolean equalsArray(Object[] array, Object[] otherArray)
   throws NullPointerException {
  if (array.length != otherArray.length) {
   errorLog.add("Array or Collection length are not equals[" + array.length + "," + otherArray.length + "]");
   return false;
  }

  ArrayList<Integer> usedIndexes = new ArrayList<Integer>();
  for (int i = 0; i < array.length; i++) {
   Object currObject = array[i];
   boolean found = false;
   for (int k = 0; k < otherArray.length; k++) {
    if (!usedIndexes.contains(k)) {
     found = doObjectEquals(currObject, otherArray[k]);
     if (found) {
      usedIndexes.add(k);
      break;
     }
    }
   }
   if (!found) {
    return false;
   }
  }

  return true;
 }

 private static boolean hasEquals(Class c) {
  Method[] methods = c.getDeclaredMethods();
  for (Method m : methods) {
   if (m.getName().equals("equals") &&
     m.getParameterTypes().length == 1) {
    return true;
   }
  }
  return false;
 }

        /*------------TEST------------*/
 public class TestClass {
  public class TestNestClass extends TestClass {
   public TestNestClass(int a, int b, ArrayList list) {
    super(a, b, list);
   }

   public void setD(int d) {
    this.d = d;
   }

   public void putMap(String s, Integer i) {
    map.put(s, i);
   }

   private int d;
   private Map<String, Integer> map = new HashMap<String, Integer>();
  }

  private int a;
  private int b;
  private ArrayList list = new ArrayList();
  private String c = "Hello!!!";

  public TestClass(int a, int b, ArrayList list) {
   this.a = a;
   this.b = b;
   this.list = list;
  }

  public int getA() {
   return a;
  }

  public int getB() {
   return b;
  }

  public String getC() {
   return c;
  }

  public void setC(String c) {
   this.c = c;
  }

  public ArrayList getList() {
   return list;
  }

  public TestNestClass createNest() {
   return new TestNestClass(a, b, list);
  }
 }

 public TestClass createTestClass(int a, int b, ArrayList list) {
  return new TestClass(a, b, list);
 }

 public static void main(String[] args) {
  ArrayList ar1 = new ArrayList();
  ar1.add(1);
  ar1.add(5);
  ar1.add(7);
  ArrayList ar2 = new ArrayList();
  ar2.add(1);
  ar2.add(5);
  ar2.add(7);
  WeakEqualsHelper eh = new WeakEqualsHelper();
  TestClass tc1 = eh.createTestClass(1, 15, ar1);
  TestClass tc2 = eh.createTestClass(1, 15, ar2);
  TestClass.TestNestClass tnc = tc1.createNest();
  TestClass.TestNestClass tnc2 = tc2.createNest();
  tnc.putMap("Hello", 1);
  tnc2.putMap("Hello", 1);
  tnc.putMap("goodbye", 1);
  tnc2.putMap("goodbye", 1);
  System.out.println("tc1.equals(tc2) == " + WeakEqualsHelper.doEquals(tnc, tnc2));

 }
}

I add some features in this implementation:
-I can pass Map<string, List<String>> that contains class name as key and list of fields name that will be ignored when equals
-If string are not equals, then its will be equals igonring escape symbols and whitespaces.
-I add simple test.

Usage: I use this class in unit testing when migrate from digester to castor. We need serialization of objects.
I rewrite over 1MB of mappings. I was in hell :)

UPD: do formatting, now you can simple copy class and use it.