56 Replies Latest reply on Mar 17, 2017 8:21 AM by Jeff Geller

    Print jobs failing to release with our Pharos mobile server

    Joseph Marasco Newbie

      We have three production servers for Pharos and also staging servers that mirror what the production servers do. One is the Principal server, second is the Student print server, and the third is the Mobile print server. I did all of the Pharos updates, service patch, and applied the mobile print 2.2 with our staging servers. Everything works as it should when doing email, web upload, and mobile upload and release to the network printers for testing. So, after a week of print testing and everything has passed, we decided to patch our production server. Everything has been updated to on our production servers.

       

      After a few weeks and more clients were utilizing the print systems, PharosMP print spooler started to stop processing the print jobs. I would have something like 72 print jobs waiting to be release on the production mobile print server's print manager. So, Pharos print jobs seem to be failing with: "Error releasing print job: Job failed to release on a Printer". The error code found via the admin portal is 29062. The release started to work again only after the print spooler has been re-started. But, after a day it would stop sending the print jobs out. Called support and we went through the Mobile print log files. We scanned through the event logs and found errors with the Microsoft-Windows-WMI namespace. The error logs attach with this post.

       

      Has anyone else been having the same issue and resolved this? The only workaround was to do a clean up script on the MOF files on the Mobile print server.

        • Re: Print jobs failing to release with our Pharos mobile server
          Mark Robertson Ranger

          We have had the same problem here at VA Tech.  Classes started and we

          started seeing the errors. I have reported it also.  No fix as of yet.

          Seems restarting the spooler will allow it to run for a short period with

          few problems then it starts cascading.

          • Re: Print jobs failing to release with our Pharos mobile server
            Gene Mayro Tracker

            We just opened an incident on this, too. What we find is that on the affected print server, Spoolsrv stops, with the PharosSystems.Cloud.ForwardingMonitor64.dll as the faulting module. We can get Mobileprint working again by stopping the Mobileprint Worker service on the affected print server, start the Print Spooler service, wait about 5-10 seconds for it to clear the backlogged jobs, then start the Mobileprint Worker service. This isn't a permanent fix, but it allows Mobileprint jobs to be released again.

             

            Gene

              • Re: Print jobs failing to release with our Pharos mobile server
                Gene Mayro Tracker

                These are the two entries we see in our Windows Applications Log:


                Faulting application name: spoolsv.exe, version: 6.3.9600.17480, time stamp: 0x54585db9

                Faulting module name: PharosSystems.Cloud.ForwardingMonitor64.dll, version: 0.0.0.0, time stamp: 0x57e8926d

                Exception code: 0xc0000409

                Fault offset: 0x000000000000da2b

                Faulting process id: 0xa88

                Faulting application start time: 0x01d271ec11e6b173

                Faulting application path: C:\Windows\System32\spoolsv.exe

                Faulting module path: C:\Program Files (x86)\PharosSystems\MobilePrint\PharosSystems.Cloud.ForwardingMonitor64.dll

                Report Id: 5eb31b22-dddf-11e6-80f8-0050568da038

                Faulting package full name:

                Faulting package-relative application ID:

                 

                 

                Followed by this error:

                Fault bucket , type 0

                Event Name: BEX64

                Response: Not available

                Cab Id: 0

                 

                Problem signature:

                P1: spoolsv.exe

                P2: 6.3.9600.17480

                P3: 54585db9

                P4: PharosSystems.Cloud.ForwardingMonitor64.dll

                P5: 0.0.0.0

                P6: 57e8926d

                P7: 000000000000da2b

                P8: c0000409

                P9: 0000000000000005

                P10:

              • Re: Print jobs failing to release with our Pharos mobile server
                Joseph Marasco Newbie

                Experience the same issue over the weekend. Print jobs was being held at the Mobile print spooler again. Had to restart the services. Spoke to the Pharos devolvement team concerning this issue. The new mobile print services is using a different way to handle the print job when it is being released. They walked us through to force the mechanism of printing not to be used by the new MobilePrint, But instead it will be handled by the secure release service. This was the previous version method of MobilePrint.

                 

                1. On the Mobile Print Workflow server, locate and edit the file: C:\Program Files (x86)\PharosSystems\MobilePrint\Plugins\SecureRelease20\PharosSystems.SecureRelease.Plugin.MobilePrint.dll.config

                2. Set the line <JobDelivery useSR="true" /> to <JobDelivery useSR="false" />

                3. Stop the Secure Release service on all printer servers.

                4. Delete the Mobile Print plugin from C:\ProgramData\PharosSystems\SecureRelease\JobOperationPlugins\MobilePrint (file:///C:/ProgramData/PharosSystems/SecureRelease/JobOperationPlugins/MobilePrint)

                5. Restart the Secure Release services.

                6. Restart the Mobile Print Workflow service.

                 

                I will continue monitor the Mobile print release and find out if this change resolves the issue.

                  • Re: Print jobs failing to release with our Pharos mobile server
                    Jeff Geller Guide

                    Hi Joseph,

                     

                    I want to make a small correction to your note above. In Mobile Print 2.2 the Pharos Secure Release Service now handles "Job Delivery" and not Mobile Print. The adjustment you mention is to tell Mobile Print to use the pre 2.2 version of Job Delivery by doing it itself and not using the Secure Release Service to deliver a Mobile Print type job (emailed or uploaded).

                     

                    From the New Features Guide for Mobile Print 2.2.

                     

                    MobilePrint now uses Secure Release print delivery

                    Starting with MobilePrint 2.2, MobilePrint jobs now use the Secure Release Service to handle job delivery (just like secure jobs). With the Secure Release handling job release, MobilePrint jobs can now be released using the ‘RAW’ print protocol. Previously, MobilePrint jobs were always sent via LPR, regardless of the protocol (e.g. RAW, LPR) the device was set to use (in Pharos Administrator), and regardless of where the job was released from. This was limiting for sites that were using printers with better support for RAW printing.

                    After installing or upgrading to MobilePrint 2.2 or later, MobilePrint jobs will use the delivery mechanism that a device is set to use (either LPR or RAW).

                     

                    -Jeff Geller

                  • Re: Print jobs failing to release with our Pharos mobile server
                    John Siegel Guide

                    Hi All,

                     

                              We are seeing the same error here at CU since school started. Restarting the spool service is the only work around so far.

                     

                    Regards,

                     

                                 John

                      • Re: Print jobs failing to release with our Pharos mobile server
                        Jeff Geller Guide

                        Hi John,

                         

                        If your seeing the same message in the Windows Event logs below then know that development is looking into this.

                         

                        Faulting application name: spoolsv.exe, version: 6.3.9600.17480, time stamp: 0x54585db9

                        Faulting module name: PharosSystems.Cloud.ForwardingMonitor64.dll, version: 0.0.0.0, time stamp: 0x57e8926d

                        Exception code: 0xc0000409

                         

                        *There is another behavior being seen that is not tied to any one version of Mobile Print and this is when the MobilePrintForward Port starts to back up with print jobs until all 10 ports are consumed (default is 10 ports). This behavior stems from a print job that fails during spooling for some reason and the Windows Print Spooler is not able to purge it and move on successfully.

                         

                        -Jeff Geller

                      • Re: Print jobs failing to release with our Pharos mobile server
                        John Siegel Guide

                        Anyone try uninstalling and then reinstalling the Mobileprint worker service? I'm going to give that a shot. So far we are seeing the error on the print server that supports all our color and MFP devices.

                        We had a problem last semester with Mobo jobs not simplexing. Reinstalling the worker service fixed it. Not that the two are related, only that we have had issues with the worker service corrupting in the
                        past. I'll let you know the results.

                         

                        ~John

                          • Re: Print jobs failing to release with our Pharos mobile server
                            John Siegel Guide

                            Hi All,

                                     Despite having restarted the spoolers on the various servers, we were getting reports across the campus that all printing was failing. I restarted all of our servers, and it appears that now everything is back to normal for the moment. What Jeff mentioned makes sense, it works until all the ports are "stuffed" and then starts backing up. This seems to be affecting both Mobileprint and regular printing when it peaks. Anyone else seeing problems outside of Mobileprint jobs?

                             

                            ~John

                              • Re: Print jobs failing to release with our Pharos mobile server
                                John Siegel Guide

                                Anyone else seeing these types of alerts? Normally we will have maybe 25 to 30 an hour. Right now it starts off at the normal rate, then builds up until we have literally hundreds of errors a hour. At that point we start to get reports of printing failing.

                                 

                                ~John

                                 

                                 

                                error list 23 jan 2017.jpg

                                  • Re: Print jobs failing to release with our Pharos mobile server
                                    Gene Mayro Tracker

                                    Hi John,

                                    That's the error we had been seeing. Our fix was to stop both the Mobileprint Worker service and the Spooler service, then start the worker service. Check the Render Queue in your Devices and printers. After 10-20 seconds, it should clear out, then start the Mobileprint Worker service. Things should return to normal.

                                     

                                    Joseph Marasco has a workaround from the developers that we are going to try tomorrow.

                                     

                                    Gene

                                    • Re: Print jobs failing to release with our Pharos mobile server
                                      Gene Mayro Tracker

                                      Hi John,

                                      Actually, I see both errors in your alerts that we have seen before.

                                       

                                      We saw this error on our one of our print servers when for some reason, the render driver stopped processing Mobileprint jobs correctly. We would see this error for any job that was sent. We were using and older HP Universal PCL driver, and switched to a newer one (just updated it on the Render Queue Printer Properties, then restarted the Worker service), and that corrected it.

                                       

                                      The second error is the one that's described in this thread, where MobilePrint jobs are not being released, and we stop the Mobileprint Worker service, then start the spooler and wait to start the Worker service again.

                                       

                                      error+list+23+jan+2017.jpg

                                • Re: Print jobs failing to release with our Pharos mobile server
                                  John Siegel Guide

                                  Hi Gene,

                                   

                                                Thanks for the update!

                                   

                                  ~John

                                  • Re: Print jobs failing to release with our Pharos mobile server
                                    Zhiyong Tian Adventurer

                                    We have been getting the same problem here at OhioU since school started. Here is what we do to temporary make MP to work again: 1. stop the MP Worker service; 2. wait for 10 seconds (or all stuck jobs disappear); 3. restart the print spooler service; 4. wait for several seconds; 5. start the MP Worker service.

                                     

                                    Seems the problem happens for MP, but I believe also see it happened for queue based printing before.

                                     

                                    Thanks.

                                     

                                    --Tian

                                    • Re: Print jobs failing to release with our Pharos mobile server
                                      Jeff Geller Guide

                                      Hi All,

                                       

                                      The Mobile Print Development team has provided a Rollup Patch that includes a fix for the DLL issue that is being seen along with a few other items. All on this thread should have received a note from me this morning along with a link to the patch.

                                       

                                      Thanks for your patience.

                                       

                                      Regards,
                                      Jeff Geller

                                      Pharos Support

                                      • Re: Print jobs failing to release with our Pharos mobile server
                                        Vinicio Guevara Adventurer

                                        we are experiencing similar issues at CUIT

                                        Has anyone tried the Rollup Patch mentioned by Jeff Geller?

                                        • Re: Print jobs failing to release with our Pharos mobile server
                                          Zhiyong Tian Adventurer

                                          We have applied and tested in our test system in this morning and it works fine (actually we have not get any of these problems/issues in the test system anyway). Plan to apply it in our production system two days late.

                                          • Re: Print jobs failing to release with our Pharos mobile server
                                            John Siegel Guide

                                            Hi All,

                                             

                                                    Although the earlier issues appear to be resolved, I'm now having an issue with one of the print servers. About evry two hours or so the print spooler services stops. It allows a restart manually and then repeats the cycle. Anyone else having this issue?

                                             

                                            ~John

                                            • Re: Print jobs failing to release with our Pharos mobile server
                                              John Siegel Guide

                                              I completed the fix installs but I'll getting this error again. Anyone else seeing this popup?

                                               

                                              ~John

                                               

                                               

                                              Faulting application name: spoolsv.exe, version: 6.3.9600.17480, time stamp: 0x54585db9

                                              Faulting module name: PharosSystems.Cloud.ForwardingMonitor64.dll, version: 0.0.0.0, time stamp: 0x58869720

                                              Exception code: 0xc0000005

                                              Fault offset: 0x000000000000fb85

                                              Faulting process id: 0x1448

                                              Faulting application start time: 0x01d2767755212aaa

                                              Faulting application path: C:\Windows\System32\spoolsv.exe

                                              Faulting module path: C:\Program Files (x86)\PharosSystems\MobilePrint\PharosSystems.Cloud.ForwardingMonitor64.dll

                                              Report Id: a83df5d8-e26a-11e6-8109-44a84234bf8a

                                              Faulting package full name:

                                              Faulting package-relative application ID:

                                              • Re: Print jobs failing to release with our Pharos mobile server
                                                Jamie Rhine Wayfarer

                                                Adding my name to the list here, as we just started having this issue as well.  It's happened three times so far today, and the suggestions for restarting services have helped temporarily.

                                                 

                                                Could I also get a copy of the rollup patch?  Is the consensus so far that it seems to resolve the issue?

                                                • Re: Print jobs failing to release with our Pharos mobile server
                                                  Willis Dair Tracker

                                                  I wanted to throw my experience in as well.  We upgraded from 9.0 to 9.0R2 SP2 then to SP4 during the Christmas break.  We also went from MobilePrint 2.0 to 2.2.

                                                  One thing I have noticed is that the MobilePrint arm of programming does not look or behave like the Uniprint arm.  MobilePrint does not install the same way as Uniprint and does not seem to have the same polish.  I'm a little annoyed that the you have to do a complete install each time you upgrade. ie, no settings are or configuration is remembered between versions.  Good thing I took snapshots of the web pages of the administrative MobilePrint configuration before upgrading...

                                                   

                                                  After the upgrade all of our printers were running on the Pharos PCL5 rendering queue.  We used to have color printers on another PS rendering queue. This is when it started to crash as you all have experienced.  The rendering queue would get a rather large print job, typically >400MB and would stop the first thread of the spooling pool.  This thread would be in the "printing" state.  Then all 10 threads would be occupied "printing" and everything is backed up.  In order to clear it up, we would restart the Windows Print spooler service.  By the way, I have noticed once the Print Spooler dies, the print job that was being processed is in the C:\Windows\Temp directory.  The file looks like MPxxx-xx.tmp.  These leftover files caused the C: to completely fill up and run out of space...

                                                   

                                                  We moved the color printers back over to the color PS rendering queue.  Then what would happen was that a huge job in the PCL5 render queue would come in, clog the PCL5 queue and take down the print spooler so that the color render queue would also clog up.  No MobilePrint jobs would print until the Print spooler service was restarted.

                                                   

                                                  So I was thinking, we just upgraded mono printers and the bulk of my mono printers were a certain make and model.  I decided to get the specific driver for that make and model and create another render queue just for that model.  So now I have one render queue for the color printers, one render queue for the specific make and model, and one render queue for the rest of the mono printers.

                                                   

                                                  I have been running for 9 days with no crashes or restarting of the Print spooler service.  Live is good again.  I'm wondering if there is something that is interacting with the Pharos PCL5 driver that was not happening in previous versions?  We used to run with the PCL5 driver and the color PS driver only.

                                                   

                                                  Willis Dair

                                                  Santa Clara University

                                                    • Re: Print jobs failing to release with our Pharos mobile server
                                                      Vinicio Guevara Adventurer

                                                      Are you seeing timeout errors?

                                                      We are still seeing a lot of these errors:

                                                      Error releasing print job: Failed to render Job Id '00000002-0000-0000-0000-000000113872' to '123.45.67.89:515', because of timeout.

                                                        • Re: Print jobs failing to release with our Pharos mobile server
                                                          Mark Robertson Ranger

                                                          Ours have completely cleared after the patch was installed.

                                                           

                                                          On Jan 30, 2017 6:22 PM, "vg2307@columbia.edu" <pharos-ext@jiveon.com>

                                                          • Re: Print jobs failing to release with our Pharos mobile server
                                                            John Siegel Guide

                                                            Hi Vinicio,

                                                             

                                                                           We see those on a regular basis. I don't think they are part of the current issue. Could be a non supported format, or just a slow connection that literally timed out.

                                                             

                                                            In our environment we noticed a plethora of two specific errors when the spooler choked.

                                                            the first was:

                                                             

                                                            LPD service refused a "receive print job command". System reports error: The TCP/IP Print Server refused the print job

                                                            (which makes sense because the spooler isn't running)

                                                             

                                                            and

                                                             

                                                            Failed to render Job Id '00000002-0000-0000-0000-000000015210' as the render server does not have an available print queue that uses the ''Xerox WorkCentre 7835 PS' driver with Greyscaling 'Converted by Printer (Recommended)''

                                                             

                                                            We hadn't seen this error prior to the current issue. This one was strange because the 7835 it mentions is a color MFD. It is unlikely that 50 jobs in a row were sent to the color MFD and had been manually changed to print using  greyscale. Although it also showed up mentioning the Global driver using B&W  printers, the consensus was that it was a diversionary error, meaning the error message itself isn't actually related to the root problem. You see these quite frequently with hardware, for example, the printer error says "laser failure", but the actual problem is a microswitch that doesn't close because the door isn't closed all the way on the printer. In this case the "Greyscaling error was most likely related to the spooler crash\backup, vs. an actual call for greyscaling from the job itself.

                                                             

                                                            We would see literally 50 plus of each error while the spooler was down\backed up. We also saw a large number of the timeout errors, but I believe that was related to the spooler failure causing the jobs to fail, since we also see the timeout error when the system is stable.

                                                             

                                                            The strangest one I saw was: error releasing print job: Printer is too busy to print job id:(job ID number). This one pops up after restarting the spooler service, It usually shows up for a few minute and then stops. Anyone else seen this error?

                                                             

                                                            Currently after "patching", the system has been stable since last Thursday with the exception of one spooler crash yesterday afternoon. ~John

                                                            • Re: Print jobs failing to release with our Pharos mobile server
                                                              Jamie Rhine Wayfarer

                                                              Yes, we are seeing the timeouts as well, and it seems related to this issue in our case (at least based on the timing of failures we've seen).

                                                               

                                                              First we see:

                                                              Error releasing print job: Failed to render Job Id '00000001-0000-0000-0000-000000030012' to '172.55.3.106:9100', because of timeout.

                                                               

                                                              And then (same job):

                                                              Error releasing print job: An unexpected error occurred while attempting to render Job Id '00000001-0000-0000-0000-000000030012'. Error: MobilePrint.Service.Worker.Tasks.PrintManager+PdlRenderInfo

                                                               

                                                              We run our primary Uniprint services on one server, and MobilePrint services on another.  We had one failure yesterday and another today, and just restarting the Print Spooler and MobilePrint Worker service on the MP box didn't fix it...  I also had to restart the Pharos Print Server service on the primary server as well.  For us, it seems like once someone tries (and fails) to print to a specific queue via MobilePrint, normal print jobs submitted natively aren't processing correctly on those queues either.

                                                          • Re: Print jobs failing to release with our Pharos mobile server
                                                            Jason Pelletier Tracker

                                                            We are also seeing these issues at Bowdoin. They appeared just a couple of days ago. We are seeing exactly what Jamie Rhine is seeing where we receive a timeout error and then the PDL render issue. We run Uniprint on the principal and Mobile on it's own server. We are actually also seeing the issue where we have to restart the Pharos Print Server service on the principal because the whole thing just stops working. Getting access to the patch when available would be great!